Design of binary fringe patterns for high speed 3 d vision

Design of Binary Fringe Patterns for High
Speed 3-D Vision
Individual Project Final Report Document
Spring 2017
George Hadler
B417099
Project Supervisor:
Prof. Jonathan Huntley
2nd Reader:
Dr Pablo Ruiz
Loughborough University
The Wolfson School of Mechanical, Electrical & Manufacturing Engineering
16MMC500 Individual Project
B.Eng Mechanical Engineering

STATEMENT OF ORIGINALITY
“This is to certify that I am responsible for the work
submitted in this report, that the original work is my own
except as specified in references, acknowledgements or in
footnotes”.
PRINT NAME: George Hadler
SIGNATURE:
DATE: 12th
April 2017

Abstract
Through the projection of sinusoidal fringes onto objects, 3-D environments can be
recorded and understood by automated machinery for a variety of uses. For the benefit of
speed, defocused binary patterns have recently been proposed to be projected rather than
classical 8-bit greyscale fringes, but this usually comes at the cost of less accurate 3-D
results. This paper is a library of different techniques of generating and optimising these
binary frames, in the search of patterns that will provide the most accurate profilometry
when projected. Through the implementation and altering of a classical Simulated Annealing
algorithm, this paper proposes a new binary image optimisation method, which
demonstrates a RMS error of around 0.003 rad when compared to an ideal sinusoid. This is
roughly a 40% improvement compared to the results of the current state of the art.

Symbols Used Throughout This Paper
D = Blur Diameter, i.e. Point Spread Function (Pixels)
β = Period of Binary Sinusoid Fringe (Pixels), also the value used for tile length.
δ = Tile Height (Pixels)
φ = Phase (Rad)
σ = Gaussian Distribution Standard Deviation
α = Multiple of Sawtooth Frequency to Base Sinusoid Frequency for STM & SSTM Methods
e = Error Tolerance used in MLIP and Pixel Mutation
CF = Cost Function for Simulated Annealing
T = ‘Temperature’ for Simulated Annealing Cooling Schedule
PWorse = Probability of Worse Acceptance for Simulated Annealing
Δ = Patch Height/Length for MLIP Method
γ = Pixel Fill Area (Kept at 92%)
Pm = Micro-Pixels Used to Form a Single Pixel in a DMD
P = Pixel Representation, formed of n x Pm
ω = Pixel Gap Size (in Number of Pm)
Z = Focal Position
I = Intensity of Image
N = Number of Iterations (Also size of problem for Section 6.1 only)
S = Steps Required to Solve a Problem

Contents
1 – Introduction................................................................................................................1
2 – Literature Review: Projection of Fringes ......................................................................4
2.1 – Colour Map Projection Methods...................................................................................5
2.2 – Phase Shifting Algorithm...............................................................................................6
2.3 – Stripe Boundary Coding.................................................................................................6
2.4 – Digital Light Processing Projector and Development of Acquisition Speeds................8
2.5 – Beginning of Binary Defocusing Techniques .................................................................9
2.6 – Current State of Binary Defocusing Techniques & Motive for This Paper’s Research10
3 – Optical Configuration................................................................................................ 11
3.1 – The Point Spread Function and Convolution ..............................................................11
4 – Mathematical Processes............................................................................................ 14
4.1 – Blurring – Convolution.................................................................................................14
4.2 – Three & Four Step Phase Shifting................................................................................15
4.2.1 – Four Step Phase Shifting Algorithm .....................................................................15
4.2.2 – Three Step Phase Shifting Algorithm ...................................................................16
4.3 – Phase Unwrapping and Comparison to Ideal Phase ...................................................17
4.3.1 – Phase Pixel Unwrapping.......................................................................................17
4.3.2 – Temporal Phase Unwrapping...............................................................................18
4.4 – Tiling of Binary Image..................................................................................................20
4.5 – Considerations When Using Blurring Simulations.......................................................20
5 – Binary Image Generation .......................................................................................... 22
5.1 – The Randomised Bits Method.....................................................................................22
5.2 – The Sawtooth Method (STM)......................................................................................23
5.3 – The Shifted Sawtooth Method (SSTM)........................................................................24
5.4 – Results of Binary Generation.......................................................................................25

5.4.1 – Discussion of Results............................................................................................26
6 – Binary Image Optimisation........................................................................................ 28
6.1 – Complexity of Optimisation: NP Hardness..................................................................28
6.2 – Mixed Linear Integer Programming (MLIP).................................................................30
6.2.1 – Overview of MLIP.................................................................................................30
6.2.2 – Issues Associated with MLIP ................................................................................32
6.2.3 – Summary ..............................................................................................................32
6.3 – Pixel Mutation Method ...............................................................................................33
6.4 – The Theory of Local / Global Minima..........................................................................37
6.5 – Simulated Annealing....................................................................................................39
6.5.1 – Annealing Based on Probability ...........................................................................39
6.5.2 – The Pixel Swapping Method ................................................................................41
6.5.3 – Issue with Probability Based Annealing with Pixel Swapping..............................42
6.5.4 – Annealing Based on Convergence .......................................................................42
6.5.5 – Using a Three Frame Annealing Technique .........................................................44
6.6 – Results of Binary Optimisation....................................................................................47
6.6.1 – Overall Results – Discussion.................................................................................47
6.6.2 – Comparison to Literature.....................................................................................51
7 – Realistic Considerations ............................................................................................ 52
7.1 – Considering Inter-Pixel Gap.........................................................................................52
7.2 – Inter-Pixel Gap Simulations.........................................................................................54
7.2.1 – Greyscale Sinusoid with Gaps ..............................................................................54
7.2.2 – Inter-Pixel Gap Effects on Simulated Annealing Results......................................57
7.3 – Discussion of Realistic Considerations ........................................................................57
8 – Summary & Conclusions............................................................................................ 59
8.1 – Closing Summary & Conclusions .................................................................................59

8.2 – Potential for Future Work and Further Investigation .................................................60
9 – Acknowledgements................................................................................................... 61
10 – References.............................................................................................................. 62
11 – Appendix – Admin Documents for Wolfson School .................................................. 65
11.1 – Original Objective Form ............................................................................................65
11.2 – Gantt Chart (As of January 2017) ..............................................................................66

Design of Binary Fringe Patterns for High Speed 3-D Vision
Page | 1
1 – Introduction
Since the start of the 21st century, the ability to measure objects in 3-D using non-intrusive
methods has escalated with new methods being developed to do so. This is applicable in a
variety of fields, including object detection, quality control and robotic intelligence [1,2] as
examples. This is mainly achieved through a variety of digital fringe projection (DFP)
techniques, where structured light is projected onto an object and can be used to obtain
data. One of the most commonly used methods is through the projection of sinusoidal
fringe patterns and through evaluating phase changes, a surface profile can be obtained.
This approach was favourable because of the potential for high measurement speed and
accuracy [3].
The process involves projecting a
series of these fringes onto 3-D
objects, which would undergo
deformation as the geometry of the
object distorts the fringes when
viewed from an off-axis position, as
shown in Figure 1. By placing a camera
at this known position, it can record
these deformed fringes and calculate a
phase map [4,5]. By retrieving the phase
difference between what the camera
has recorded and the original, un-
deformed fringes, a computer can
obtain a surface profile of the object,
which can then be used for whatever
application from there onwards. The need for this can be found in a variety of industries.
From a robotics point of view, if a machine could fully detect its surrounding environment
and respond in real-time, it could bring endless possibilities.
Camera + Computer at angle θ
to DLP Projector
Projector
Projected
sinusoid fringes
have been
distorted due to
presence of tea
mug
Figure 1: Simple schematic used to generate 3-D images. A tea mug is
placed in-view of the camera and the DLP fringe projector and camera
records the distorted image.

Design of Binary Fringe Patterns for High Speed 3-D Vision: 1: Introduction
Page | 2
It has been emphasised that the fringes should be as close to a sinusoidal intensity as
possible for accurate results, and different projection methods have been implemented to
do so. Commonly, 8-bit pixel grey levels ranging from 0-255 were used to generate the
series of these fringes, which whilst provided accurate surface results, was a relatively slow
process. This is due to the grey-scale intensities being simulated using ‘pulse-width-
modulation’, where the pixel’s ‘on’ intensity is quickly pulsed to replicate a grey intensity.
Having to do this for a variety of 255 intensities limits how quickly a projector can display
images. With speed of measurements only reaching a max of 120Hz [6], only objects that
were stationary or slowly moving could be recorded.
To combat this lack of projector speed found with 8-bit intensities, an alternative was
proposed which generated the sinusoidal fringes using only 1-bit binary patterns. This way,
rather than the projector having to cycle between 8-bit intensities, the only pixel options
were either 1 (on) or 0 (off). Then through blurring the projected images, a quasi-sinusoidal
fringe pattern could be replicated in much quicker speeds up to 30kHz [7,8,31].
Having the fringes be as close to sinusoidal in intensity is fundamental for accurate 3-D
imaging (as shown later in this paper), but going from 8-bit intensities to 1-bit makes it
harder to satisfy this. This paper therefore examines and compares different methods of
making the ideal binary fringe pattern, either found from researched literature or self-
invented methods.
The start of this paper (Section 2) provides a brief literature review, detailing previous
methods of 3-D profilometry using structured light, evaluating their advantages and
drawbacks. This will then lead to the current state of binary DFP and explain the need for
the topics discussed later in the paper. Section 3 then details the schematic of projection
systems and how their defocus levels can be mathematically quantified.
The rest of the paper then evaluates the quality of binary patterns. Firstly, the general
mathematical processes used to carry out the analysis is specifically detailed in Section 4.
The first part of actual analysis will then be looking at how to generate binary patterns from
scratch and comparing their effectiveness at replicating a sinusoid (Section 5). This will be
followed by examining optimisation methods of binary patterns in Section 6, which takes
pre-existing arrangements and through a variety of algorithms, refines the fringes to give

Design of Binary Fringe Patterns for High Speed 3-D Vision: 1: Introduction
Page | 3
the best possible layout. This section also includes a newly proposed method based off a
Simulated Annealing optimisation principle, which has proved to give leading results
compared to those found in the literature, by approximately 40%.
The main cost function used throughout this analysis will be the phase error between the
blurred patterns and that of a perfect sine wave, through various simulations in MATLAB of
all methods.
The latter parts of this paper will discuss real-life factors that need to be considered when
implementing these techniques in reality (Section 7), such as inter-pixel gap, which will
increase phase error compared to the simplified simulated model. Finally, a brief discussion
of all results will be carried out at the end of this paper.

Page | 4
2 – Literature Review: Projection of Fringes
3-D Structured light profilometry (SLP) is fundamentally using non-contact projection
techniques to find an objects depth at several locations, which when combined can
ultimately form a three-dimensional profile. Figure 2 shows the setup of a typical digital-
fringe-projection (DFP) arrangement. By placing the projector at a known distance and angle
from the object being measured and the camera, triangulation allows 3-D data to be
obtained [8,9].
As explained in Section 1, the two main requirements of successful 3-D measurement are
accuracy and speed, so naturally different DFP methods were experimented and compared
to provide the best results. SLP techniques are normally categorised by some key variables,
including: number of frames required to be projected (i.e. single shot vs multi shot),
whether they use colour, grey-scale or binary and other factors. The following section
discusses some of the various methods of implementing SLP, including their advantages and
disadvantages in obtaining 3-D data. This then leads to the early stages of the DLP
techniques used in this paper and how the field has progressed since, ultimately leading to
the methods today.
BaselineProjected fringe pattern
Distorted fringes due to object
being placed in front of pattern Example point of
interest
Phase line of interest
Image received by
camera
θ
Figure 2: A more detailed schematic of implementing structured light for 3-D profilometry

Design of Binary Fringe Patterns for High Speed 3-D Vision: 2: Literature Review
Page | 5
2.1 – Colour Map Projection Methods
Some methods to measure surface profiles use colour as their main focal point. Li Zhang et
al[10] describes his ‘one shot’ method which has a similar setup as to that of Figure 2, but
rather than project a series of greyscale sinusoids, he would project a single pattern
comprised of lots of vertical RGB colour stripes onto the object being measured. Again, a
camera captures the resulting image and 3-D data is reconstructed based off the relative
displacement of points in the recorded image compared to that of the original projected
pattern.
The main advantage in using colour coded patterns is that it requires only one single
projection frame and all the analysis could be carried out from that single image. This results
in fast data acquisition speed, due to the projector’s pixel switching capability not being
tested. However, colour based projection techniques all have similar issues which reduce
the accuracy of results. Surface reflectivity and the colour of the object itself often results in
problems coupling the data points from the received and projected images, resulting in
inaccurate 3-D reconstruction [9,11]. This means that colour based methods can usually only
be applied in very specific, and often unrealistic, environments where ideal lighting and
objects are present. Because of these issues, it is necessary to develop fast data acquisition
processes using binary and greyscale projection methods which are not affected by the
problems associated with colour techniques.
Figure 3: Demonstration of colour map projection. Taken from Li Zhang [10]

Page | 6
2.2 – Phase Shifting Algorithm
One of the most classical ways of obtaining fringe data to reconstruct a signal is through the
Phase Shifting Algorithm. K. Creath compiled a list of equations in 1993 [33] that shows phase
data can be received by projecting several images, which are phase shifted by a certain
amount. Then, after these images have had their intensities recorded, an equation can use
this to calculate phase. The minimum number of images that can be used are 3 and can go
up from there. The equation to find phase changes with the number of frames used.
This principle was used from then on in a variety of 3-D profilometry methods, with authors
obtaining phase using multi-shot phase shifting on their fringe patterns. The equations and
full explanation of phase shifting can be found in Section 4.2.
2.3 – Stripe Boundary Coding
An early adaptation of multi-shot projection came from a method known as ‘Stripe
Boundary Coding’ [12] (SBC) in 2001. In this method, a pattern of binary stripes is projected
onto the object in question, then a second pattern is projected, but each stripe has at least
Figure 4: Stripe Boundary Coding example: taken from O. Hall-Holt et al [12]
. See how the processor can
obtain a unique code for each boundary upon data retrieval and correspond it to the original pattern.

Page | 7
one of its neighbouring stripes change in intensity (Figure 4). This is done four times (i.e. 4
different binary patterns), so each stripe has a unique list of sequential boundary changes,
meaning no two fringes in the array undergoes the same four neighbour changes.
So, once the camera receives all four images and processes them, it can pinpoint the
displacement of the fringes by finding its unique boundary code. This boundary code is
made from the unique changes undergone at the edge of each fringe after the 4 sequential
images.
This method is flawed however for several reasons, the main being that this process only
actually provides surface depth for the boundaries of the stripes, rather than the whole
fringe itself. This means that if there were any sudden discontinuities in the object’s surface
smaller than the fringe size, it would not be received by the processor. In relation to this
paper, the speed of this method is also a flaw, as the frame cycle speed was only 60Hz, then
having to project 4 images results in a maximum of 15fps acquisition speed. This limits the
SBC method to only stationary or slow moving objects and does not make it appropriate for
the ‘high speed’ applications discussed in Section 1. This method also presented issues when
objects underwent movement as the camera often failed to match the correct boundaries
to each other under speed.
Coding methods are common in 3-D profilometry and there are a variety of different
techniques apart from Stripe Boundary Coding. A common one similar to this is ‘Gray
Coding’ where illuminating a ‘binary code’ in the form of a sequence of black and white
stripes onto an object [11]. Then by matching the binary codes in the original and deformed
images, depth information can be obtained. It is also common to combine this method with
phase shifting techniques (as in Section 2.2) to utilise the benefits of both, as explained by S.
Zhang in 2010 [37]. There are several hybrid methods of coding and phase shifting available
for use depending on the requirement.

Page | 8
2.4 – Digital Light Processing Projector and Development of Acquisition Speeds
In an attempt to overcome the slow data acquisition rate, a technique was developed to
obtain a 3-D surface profile which would prove to be the foundation of following methods,
including the one presented in this paper. It took advantage of a digital light processing
(DLP) projector, which at the time was attractive for its high-speed colour pattern projection
capability. The basis of DLP projectors is the digital micromirror device (DMD), which is
made up of an array of tiny mirrors, which can rotate on an axis by angles +θL (ON) or -θL
(OFF)[14] to reflect or cut off light projected onto it (Figure 5). The exposure time in which
one of the mirrors reflects the light governs the brightness of that pixel, meaning that each
pixel can individually be controlled for whatever application.
Usually these projectors have a colour wheel which allows the projection of RGB patterns,
but for this process, this component can be removed, turning the images greyscale. This was
done in the late 1990s and early 2000s by Huang et al [3] and Coggrave [35] for examples. To
obtain 3-D surface data, three greyscale sinusoidal fringes were projected onto objects and
using the Phase Shifting Algorithm of Section 2.2, a surface profile could be obtained by
calculating the phase difference between the input fringes and the received images. At the
time when Huang did this for example in 2003, his analysis with the modified projector
allowed projection of images of up to 300Hz (100Hz cycles), however this was limited by the
capabilities of his recording camera, which only process images at 85Hz. This meant the
camera could only record images every other full projection cycle, resulting in a 3-D
acquisition speed of only 16fps[3,11].
Given that this breakthrough only occurred just around the millennium, naturally people
would adapt this technique and modify components further to provide a faster, more ‘real
Figure 5: Schematic of a single DMD DLP Projector. Taken
from Huang et al [3]

Page | 9
time’ response as technology developed. By 2010, Liu et al [13] had invented a technique
which obtain 3-D data at a rate of 20fps (6 frames at 120Hz) with his ‘Dual Frequency’
method. He could obtain this level of projection speed by using the Texas Instrument
Discovery 1100 DMD board in the projector, which allow pre-set 8-bit and binary patterns to
be projected at 120Hz and 4000Hz respectively [11].
2.5 – Beginning of Binary Defocusing Techniques
The previous section explains that using a grey-scale technique, 3-D data could be obtained
at around 20fps. For many applications, this would be fast enough to accomplish tasks
involving stationary or slow moving objects. The applications mentioned in Section 1 could
require acquisition speeds of a magnitude higher than this. This is where the use of binary
projection truly comes into play, combining it with the DLP techniques used by the likes of
Huang and Coggrave. Gong et al [14] proposed a method in which a single shot binary pattern
is to be projected. In this arrangement, each pixel could only be set to either 1 (ON) or 0
(OFF) and through blurring of the projected images, it could closely replicate that of a
sinusoid intensity profile obtainable with an 8-bit projection scheme. This resulted in 3-D
data to be obtained at 4000Hz, much better than previously proposed methods. However,
this single shot technique utilised an inverse Fourier transform to find phase 𝜙 of the
received image using the following equation:
𝜙(𝑥, 𝑦) = 𝑎𝑟𝑐𝑡𝑎𝑛 {
𝐼𝑚[𝐼𝑓(𝑥, 𝑦)]
𝑅𝑒[𝐼𝑓(𝑥, 𝑦)]
}
with 𝐼𝑓 being the Fourier domain of received intensity.
Gong explains that this method requires precise calibration in order to provide accurate 3-D
data, which proves to be challenging and often unlikely when using working with blurred
images. Ultimately this rendered this technique fast, but with often inaccurate results. From
this it was learned that binary methods could be the best avenue of investigation to find a
fast method of 3-D measurement, the difficult part therefore was to find a way of doing so
which still provided accurate results.
(1)

Page | 10
2.6 – Current State of Binary Defocusing Techniques & Motive for This Paper’s Research
To overcome the issues mentioned above with the single shot binary blurring technique,
methods were proposed which used the Phase Shifting Algorithm, as briefly explained in
Section 2.2. The most common solution is one which projects three binary images, that
undergo deliberate blurring to approximate to a sinusoidal intensity and through the use of
the ‘Three Step Phase Shifting Algorithm’ [6, 7, 14, 15, 16, 17, 18, 19, 20, 33] utilising the following
equations allows phase to be found:
𝐼1(𝑥, 𝑦) = 𝐴(𝑥, 𝑦) + 𝐵(𝑥, 𝑦) cos (𝜙 −
2𝜋
3
)
𝐼2(𝑥, 𝑦) = 𝐴(𝑥, 𝑦) + 𝐵(𝑥, 𝑦) cos(𝜙)
𝐼3(𝑥, 𝑦) = 𝐴(𝑥, 𝑦) + 𝐵(𝑥, 𝑦) cos (𝜙 +
2𝜋
3
)
where 𝜙 is the phase, A and B are the background intensity and modulation intensity
respectively.
Solving this gives:
𝜙(𝑥, 𝑦) = 𝑎𝑟𝑐𝑡𝑎𝑛 (
√3 ∙ (𝐼1 − 𝐼3)
2𝐼2 − 𝐼1 − 𝐼3
)
From Eqn. 5, the phase 𝜙 can be obtained by solving the three intensity patterns In. This
equation will be explained in more depth in Section 4.2.
As DLP technology progressed, 1-bit binary images were able to be switched at a rate of tens
of kHz [18] which could theoretically mean using a 3-step algorithm, could achieve
measurement speeds of kHz. In 2010, Zhang et al used this method and could achieve an
unprecedented 3-D acquisition speed of 667Hz [16]. With acquisition speeds reaching this
‘ultrafast’ level, the next main challenge is to find what binary pattern arrangement should
be implemented that produces the closest to sinusoid intensity profile, being the basis for
the rest of this paper.
(2)
(3)
(4)
(5)

Page | 11
3 – Optical Configuration
This section details the schematic of how light is projected and how applying appropriate
defocus can be mathematically quantified. This will aid the understanding of the methods
used in Section 4 onwards.
3.1 – The Point Spread Function and Convolution
In the setup used for this paper, some alterations must be made to the projection system in
order to implement appropriate blurring for the binary patterns in use. Figure 6 below
shows a schematic projector diagram implementing a DMD:
From Figure 6, the ideal focus point P from the light source appears to spread around its
intended position upon defocus, forming a circle of diameter Dx with its centre point P. The
amount by which this point ‘expands’ is known as the ‘Point Spread Function’ (PSF), relating
the diameter Dx to the point P and this is ultimately the variable in which projector blur can
Input source point
light
DMD
Projection
Lens
Best focus output
Output image
from blurring
Output image
from blurring
D1
Point, P
D2
Z
Figure 6: A projector schematic. The solid middle shape represents the output light from a
perfectly focused projector. The dashed ones are the output shapes from defocused
projection. The light is a function of Z, the degree of blur/defocus. Upon different degrees of
blurring, the optimal focused point P becomes spread to the range Dx

Design of Binary Fringe Patterns for High Speed 3-D Vision: 3: Optical Configuration
Page | 12
be quantified. This can be achieved through several ways, including regular defocusing of
the projector or including geometrical changes to components, such as a spherical
aberration lens. A graphical representation of D as a function of Z can be found in Figure 8.
In a simulation, the PSF blurring can be represented by what is known as ‘convolution’,
which can be imagined as a window which moves across an array, taking a moving average
of the intensities it comes across. In the case for this paper, the ‘window’ size would be the
PSF diameter, D and the array it takes a moving average of would be the binary image,
therefore providing a blur effect.
Figure 7: Diagram of PSF. The ideal focus point P is spread by diameter D
D
Figure 8: PSF Diameter vs Focus Position,
Z. Includes scenario with and without
geometrically altered lens
Position of best focus
Figure 9: Two types of convolution: ‘Top Hat’ (left) and
Gaussian (right + Eqn. 6)

Design of Binary Fringe Patterns for High Speed 3-D Vision: 3: Optical Configuration
Page | 13
The two most common types of convolution kernel (Figure 9) are ‘top hat’, commonly
associated with defocus and Gaussian functions, associated with lens aberration. Either of
which can be used in simulations to approximately quantify blur in real life scenarios.
Papers most tend to use Gaussian blur for their calculations [7, 15, 17, 18, 20, 21, 22].
The general equation for a Gaussian distribution is as follows [23]:
𝐼 =
1
√2𝜋𝜎2
𝑒
−
(𝑥−𝜇)2
2𝜎2
Combining all the points in this section, it is worth noting to finish off that there are quite a
lot of variables and considerations to take into account when representing a real-life
scenario in simulations. The PSF for example would be very difficult to perfectly derive for a
spherical lens undergoing defocus and may not match the trend of the Gaussian filter used
for simulations. It is because of this that the solutions in this paper will not be exactly
replicated when implemented for a real life DLP, and a degree of uncertainty is appropriate
to consider. There are several realistic characteristics of projectors and DMDs that are
neglected for simulations. Section 7 details some of these and the magnitude of difference
including them has on results.
(6)

Page | 14
4 – Mathematical Processes
Before any analysis of binary patterns can be investigated, some general relevant
mathematical techniques need to be explained first. The following algorithms detail how the
blur of images can be quantified along with the comparison of phase. This allows the user to
evaluate the quality of phase of the blurred binary pattern to that of an ideal sinusoid
statistically, which can ultimately display whether it is a good approximation or not. These
will be the processes used throughout the rest of the paper.
4.1 – Blurring – Convolution
As explained in Section 3.1, there needs to be a mathematical way to simulate blur of the
binary patterns. Picturing the convolution kernel as a sliding window which takes a moving
average of the 2-D binary image, as the kernel progresses over the image, a blurring effect is
made. Figure 10 shows the kernel types as they would be used for this application and
Figure 11 shows an example of their blurring results. There are several variables to consider
when deciding what kernel to implement, including dimensions, shape and standard
deviation, σ (Gaussian only).
Figure 10: 3-D representation of the two convolution kernels, like that of Figure 9
Figure 11: Original binary image (centre), Top-hat convolved image (left) and Gaussian
convolved image (right)

Design of Binary Fringe Patterns for High Speed 3-D Vision: 4: Mathematical Processes
Page | 15
4.2 – Three & Four Step Phase Shifting
The objective of blurring and analysing the patterns is to find their phase and compare it to
that of an ideal sinusoid, however this is not possible from using a single image. The phase
shifting algorithm is a popular method used to achieve phase data, which works by
generating several images with an equal phase shift. The higher number of shifted images
results in a more accurate calculation of phase, as proved in simulations, but in a real-life
scenario the aim is to use the minimum number of frames to achieve fast 3-D acquisition
speeds.
4.2.1 – Four Step Phase Shifting Algorithm
This method utilises four frames, all shifted by π/2, achieving a complete cycle after 4
images. The phase is calculated from the following
equation:
𝐼1(𝑥) = 𝐴 + 𝐵𝑐𝑜𝑠(𝜙(𝑥)) = 𝐴 + 𝐵𝑐𝑜𝑠(𝜙)
𝐼2(𝑥) = 𝐴 + 𝐵𝑐𝑜𝑠 (𝜙(𝑥) +
𝜋
2
) = 𝐴 − 𝐵𝑠𝑖𝑛(𝜙)
𝐼3(𝑥) = 𝐴 + 𝐵𝑐𝑜𝑠(𝜙(𝑥) + 𝜋) = 𝐴 − 𝐵𝑐𝑜𝑠(𝜙)
𝐼4(𝑥) = 𝐴 + 𝐵𝑐𝑜𝑠 (𝜙(𝑥) +
3𝜋
2
) = 𝐴 + 𝐵𝑠𝑖𝑛(𝜙)
where Ix is the intensity at a point along 𝑥 and 𝜙 is the
corresponding phase. Combining these equations gives the simplification:
𝐼4 − 𝐼2
𝐼1 − 𝐼3
=
2𝐵𝑠𝑖𝑛(𝜙)
2𝐵𝑐𝑜𝑠(𝜙)
= tan(𝜙)
which allows the final phase along x to be calculated by:
𝜙(𝑥) = 𝑎𝑟𝑐𝑡𝑎𝑛 [
𝐼4 − 𝐼2
𝐼1 − 𝐼3
]
with the phase calculated, it can then be compared to that of an ideal sinusoid, as
demonstrated in Section 4.3.
(7.1)
(7.2)
(7.3)
(7.4)
(7.5)
(7.6)
Figure 12: 4 Phase Shifted Images
𝑥
𝑥
𝑥
𝑥

Page | 16
4.2.2 – Three Step Phase Shifting Algorithm
As noted in Section 2.6, for the benefit of speed, in real life applications it is preferred to use
three frames shifted by 2π/3, being the minimum required number of images to accurately
calculate phase and is used more commonly in literature [6, 7, 17, 18, 19, 20, 24]. It uses the
following equations:
𝐼1(𝑥) = 𝐴 + 𝐵𝑐𝑜𝑠 (𝜙(𝑥) −
2𝜋
3
)
𝐼2(𝑥) = 𝐴 + 𝐵𝑐𝑜𝑠(𝜙(𝑥))
𝐼1(𝑥) = 𝐴 + 𝐵𝑐𝑜𝑠 (𝜙(𝑥) +
2𝜋
3
)
Then solving these three equations like that of Section 4.2.1, the phase is calculated:
𝜙(𝑥) = 𝑎𝑟𝑐𝑡𝑎𝑛 [
√3(𝐼1 − 𝐼3)
2𝐼2 − 𝐼1 − 𝐼3
]
From Eqn. 8.4, you can see that phase is derived using one less required image to that of
Eqn. 7.6, allowing for faster 3-D acquisition speed. As this is more commonly used and
applicable to this project, the three-step algorithm will be the one used for the duration of
this paper.
(8.1)
(8.2)
(8.3)
(8.4)

Page | 17
4.3 – Phase Unwrapping and Comparison to Ideal Phase
4.3.1 – Phase Pixel Unwrapping
When using the phase shifting algorithm, because of the use of the arctangent function,
there will be sudden discontinuities in the ‘wrapped’ phase output. To overcome this,
‘phase unwrapping’ is used, which is simply the addition of ±2π where a sudden phase jump
of π or greater is encountered, as shown in Figure 13.
Once the unwrapped phase has been obtained, it can be easily compared to the ideal sine
phase to determine the quality of the binary pattern. Using Figure 14, by computing the
difference between the true and unwrapped phase at each point (Eqn. 9), an error plot can
be obtained and a single RMS value of error can be obtained for this particular pattern. The
overall RMSE of the binary pattern is the main cost function of this paper, with the aim
being to try to minimise this value. This single value ultimately governs how well the blurred
pattern approximate a sinusoidal intensity.
Δ𝜙(𝑥, 𝑦) = 𝜙 𝐵𝑙𝑢𝑟𝑟𝑒𝑑 (𝑥, 𝑦) − 𝜙 𝑇𝑟𝑢𝑒(𝑥, 𝑦) (9)
Figure 13: a) Unwrapped Phase, because of the Phase Shift Algorithm, b) Phase after unwrapping
a) b)
𝑥 𝑥

Page | 18
4.3.2 – Temporal Phase Unwrapping
When measuring 3-D objects, using three images works well when the object being
recorded is continuous in its surface profile, however this isn’t always the case. Usually
when using a single set of images of the same frequency, the presence of noise and
geometrical discontinuities can cause the sharp 2π shifts to propagate throughout the image
[35, 36]. To overcome this, ‘Temporal Phase Shifting’ can be applied. In this method, there are
a range of fringe frequencies that are projected as time progresses, for example frequencies
of 32, 31, 30, 28, 24, 16, giving 6 different fringe patterns. If using three step phase shifting,
this provides 18 patterns that should be projected.
So, suppose an object with discontinuities in its surface profile with the different fringe
frequencies being projected onto it. For each of these frequencies, the pattern is shifted as
usual and the wrapped phase 𝜙 is found as per Eqn. 8.4. However, the phase cannot be
Figure 14: Comparing the phase of a randomised bits (Section 5.1) binary pattern with a Gaussian blur. RMS Error between the
two is approximately 0.04.

Page | 19
unwrapped from one pixel to the next, as you cannot tell if there is a discontinuity between
them. Repeating this process for example using the 6 frequencies mentioned previously,
each pixel will have 6 values of wrapped phase, allowing the user to plot phase vs fringe
number [35, 36]. This will provide a 1-D signal of phase, similar to that of Figure 14, but in this
case, it is a function of fringe number, not pixels across the image. Now that the phase
values all come from one single pixel, it will not matter if there are object discontinuities
between pixels.
Clearly the more projected images used, the less speed of data acquisition, so it is therefore
necessary to use temporal unwrapping to plot the phase map using the minimal amount of
fringe frequencies. It was discovered that intermediate fringe numbers can be skipped in
projection with the phase plot still able to be plotted [35], so rather than projecting fringes 1,
2, 3, 4 etc, you can project 1, 2, 4, 8, 16… (exponential) or …16, 8, 4, 2, 1 (reverse
exponential) to save on projection and acquisition speeds, still allowing a fast response.
In relation to this paper, this demonstrates why it is necessary to generate and optimise
fringes with a range of frequencies, not just a single frequency of fringe. If a low RMSE for
patterns can be obtained for several fringe numbers using the methods as such in Section
4.3.1, then when temporal unwrapping is applied for real-life use, the 3-D data will be of
high accuracy. For demonstration in this paper, the fringe frequencies (variable β), in pixels
will be 24, 48, 72, 96, 120.

Page | 20
4.4 – Tiling of Binary Image
When generating binary patterns and optimising them, a clear assumption to be made is
that the fringes are exact and periodic. In other words, the nth fringe should be identical to
the (n+1)th fringe. So, theoretically the unwrapped phase should be of the same RMS error
between that and an ideal sinusoid if 1 or say 10 fringes were used. Taking advantage of
this, for computational and simplicity sakes, the phase of only one period of fringe will be
analysed and altered. Then for generating the whole image, this optimised pattern will be
tiled across a grid to produce a full resolution image, such as in Figure 15. The length of the
tile is to be period of the fringe (variable β). The height of the tile (variable δ) however is to
be decided by the user. For optimisation in Section 6, a small tile height compared to length
is ideal (such as δ = 12, 16, 20, 24) for computational sakes and are used in this paper.
4.5 – Considerations When Using Blurring Simulations
When using convolution in a simulation (MATLAB for example), it starts from the furthest
edge of the array and in a sense, must ‘build up’ the moving average from 0. When at the
boundaries of the array, a large proportion of the kernel will be ‘off’ the image, meaning it is
multiplying by 0. Figure 16 shows a simple example of this, with the convolved binary image
having to build up from the boundaries before a full average can be taken.
Figure 15: Example of a single tile of binary arrangement (left), repeatedly tiled onto the right image.
Tile: δ = 24, β = 144. Whole Image: Ny = 384, Nx = 1008

Page | 21
This can prove a major problem
when working out the phase of
a binary image through
convolution, because the
boundaries will cause dark
edges around the blurred image.
These dark regions greatly
increase the RMS error of the
image and should be avoided.
To overcome this, it is wise to extend the binary image by replicating it on a 3x3 grid,
convolving this whole frame and then only considering the phase of the middle section.
Figure 17 demonstrates this graphically. In carrying out this procedure, the edge effects of
convolution are well out of the way of the area of interest, therefore not creating the errors
associated with it.
Figure 16: Example of the boundary effect of convolution
Figure 17: Example of extending a binary image
a) Binary image in question, b) Extended binary image with original in
centre of frame, c) blurred extended binary image (top hat, D = 10), d)
Intensity of middle row. Only analysis of the region between the two
dashed lines is considered, neglecting the clear edge effects at either
side of the plot.
a) b) c)
d)

Page | 22
5 – Binary Image Generation
The first step of finding the optimal binary image is to actually generate the pattern from
scratch. There are many inventive ways one could do this and different methods provide
better replication of sinusoids than others. It is key to obtain the best possible results at this
stage before any optimisation occurs, as it provides the best chance of finding the ideal
solution in Section 6. The following section explains three different methods of generating
fringe patterns, demonstrating how they work and ultimately comparing their effectiveness
at developing a sinusoidal phase.
5.1 – The Randomised Bits Method
This first method is a self-invented one that uses a randomised distribution of bits based off
an ideal sinusoid. To imagine this, suppose a sine wave with its y axis ranging in integer
values from 0 to the height of the image (δ) and its x values integers from 0 to the fringe
period (β). At every position of x, let the corresponding y value equal the number of white (1
– ON) bits. So for example a coordinate (20,17) would mean at x = 20, there are 17 1s (on)
and (δ – 17) 0s (off). As shown in Figure 18, all the pixels within each column are then
randomly distributed along the height of the frame, generating a binary image of
dimensions δ by β.
Figure 18: Demonstration of the Randomised Bits method. Note the number of black pixels in b) corresponds
to the same amount of black pixels at that value of x in a). δ has been made relatively high in this image for
demonstrational purposes
a)
b)

Design of Binary Fringe Patterns for High Speed 3-D Vision: 5: Generation of Binary Patterns
Page | 23
The intensity of the sine wave for the randomised bits is the following function:
𝐼(𝑥) = sin(𝜙 𝑇𝑟𝑢𝑒)
Upon completion of creating this arrangement, it can be blurred with a convolution kernel,
then undergo phase shifting and unwrapped (as in Section 4) to find the phase and RMSE.
For the randomised bits method, the only variables to be considered for it are fringe period
and tile height, β and δ respectively. The results of this method can be found in Section 5.4.
5.2 – The Sawtooth Method (STM)
The second method in this section is taken from Beiwin Li et al [18] which involves comparing
two signals together and forming a sinusoidal pattern as a result. It works by plotting two 1-
D signals on the same set of axes, one being a perfect sinusoid and the other a ‘sawtooth’
wave. Then for each value of x along the signal, if the sinusoid is of higher intensity than the
sawtooth wave at that point, a 1 (on) is plotted on a separate graph and a 0 (off) if it is
lower. As displayed in Figure 19, this provides a 1-D pattern which can then be swept down
across a 2-D frame providing a binary fringe arrangement. Li also suggests that the sawtooth
wave’s frequency component must be an integer multiple of the sine wave’s frequency,
shown in Eqn. 11. Because of this, a new variable is included in the process, being this
integer multiplier (called α in this paper), adding an extra level of complexity for finding the
optimal pattern.
𝑓𝑆𝑎𝑤𝑡𝑜𝑜𝑡ℎ = 𝛼𝑓𝑆𝑖𝑛𝑢𝑠𝑜𝑖𝑑
Again, the results of this pattern can be found with the others in Section 5.4.
(10)
(11)

Page | 24
5.3 – The Shifted Sawtooth Method (SSTM)
Whilst the STM method of Section 5.2 successfully creates a 2-D binary image by sweeping
down a 1-D one, the result will however only be made of complete vertical bars. This may
lead to quite a strong amount of blurring to be implemented to sufficiently blend the image.
It is therefore interesting to adapt this method so that creates a binary pattern that is not
continuous as it swept down.
This leads to the creation of a newly proposed generation method, called the ‘Shifted
Sawtooth Method’ (SSTM), which starts off the same as the regular sawtooth method
(STM). Comparing the two signals again and creating a 1-D binary image is the first step, but
then the sawtooth wave is shifted by a small amount of phase whilst keeping the base sine
wave fixed. The process is then repeated forming a 2nd 1-D binary sequence, then the
sawtooth is shifted again and so on. This is done until the whole image can be populated
with these unique 1-D binary signals. So, using an example of δ = 24 pixels and β = 64 pixels,
the algorithm would generate 24 individual binary signals of length 64 and then fill the
image with these patterns. Using this routine and a sawtooth shift of π/16, it can form the
image shown in Figure 20. This method has the same number of variables as the STM does,
being β, α, and δ.
a)
b)
c)
Figure 19: The Sawtooth Method. a) Sinusoid + sawtooth signal. b) Corresponding binary arrangement. c) Swept
2-D binary image

Page | 25
5.4 – Results of Binary Generation
The final part of this section is comparing the quality of the three techniques mentioned
previously. To analyse their effectiveness, several tests have been run on each image. This
includes experimenting with different values of period β, frequency multiplier α (for STM
and SSTM), blurring type (i.e Gaussian or Top Hat) and kernel dimensions.
Figure 20: The Shifted Sawtooth Method. An example image with δ = 72, β = 48
Figure 21: Examples of the three binary image generation techniques

Page | 26
The graphical results of the analysis are found in Figure 22. They cover all three methods
being used with β values 48, 72 & 96, keeping δ constant at 24 (Because this height will be
used in Section 6’s optimisation algorithms. Each have been applied with both top hat and
Gaussian blurs separately, with blur radius ranging from 9 to 31 in intervals of 2 (σ for
Gaussian is constant at 13/3).
5.4.1 – Discussion of Results
From looking at the results on the following page, it is clear there are some general trends
being followed. Overall for every case, increasing the strength of the blur does provide
better results, as this blends the binary pattern more into a quasi-sinusoidal intensity. The
graphs do make it unambiguous that the Randomised Bits method provides the best results,
for nearly all values of β. It is curious how the STM method starts off rivalling the RB
method, but then rises considerably in error to the point it is not even within the axis range.
This could make sense however, as Figure 21 shows the STM binary image to be primarily
formed of large regions with no intensity change, meaning a very strong blur would be
required to sufficiently blend the pattern to form a sinusoidal intensity. At small β values, a
small blur radius can overcome this as the vertical bars are relatively closer to each other.
The SSTM method provides interesting results also. It is quite inconsistent in its
performance, sometimes being the near minima of RMSE, then in other tests being one of
the worst performers. The value of α seems to be more crucial in this method too, as
changing it appears to have large effects on overall phase error.
In this application, it is not ideal to apply too strong of a blur to these images, as it can lead
to contrast errors [25]. This would especially be the case if the fringes were to become
orthogonal in any way, as the blur kernel could end up equalling the pitch of the fringe,
furthermore causing an overall contrast of 0. Because of this, it is ideal to have a binary
generation method that converges in its RMSE as soon as possible at low blur values. From
the collected data, it shows the Randomised Bits method is the clear overall leader in
achieving this and shows promising results before the optimisation techniques in the
following section.

Page | 27
Figure 22: Graphical Results of generation
Left: Gaussian Blur. Right: Top Hat Blur
β increases from top to bottom (NB: For comparison sakes, the axis ranges have been kept constant, neglecting some high
values from the STM method at higher β values)
Min RMSE ≈ 0.01 @ β = 72, Gaussian Blur, radius > 15

Page | 28
6 – Binary Image Optimisation
Whilst the results from Section 5 provide promising results in reaching a close to sinusoidal
approximation, there is still more that can be done to reduce the RMSE even further. There
are several algorithms which can take pre-existing binary patterns, examine and then
repeatedly refine them aiming to achieve the absolute best possible solution. This following
section compares a number of these optimisation processes, using the data and binary images
obtained from Section 5. These are mix of pre-existing and new refinement methods which
should provide the lowest RMS phase error possible.
6.1 – Complexity of Optimisation: NP Hardness
Before optimisation methods are discussed, the need for them should be explained first.
Computers are renowned for being able to complete basic mathematical operations almost
instantaneously, such as multiplication, and complexity of these problems depend on the size
of the problem.
Computational problems are usually split into two categories:
1) Polynomial Time Problems (P)
Polynomial time problems are those where the number of steps required to solve a
problem, S, is a polynomial function of the problem’s size, N, such that:
𝑆 = 𝑓(𝑁)
𝐸𝑥𝑎𝑚𝑝𝑙𝑒: 𝑆 = 3𝑁2
+ 𝑁 − 2
With the above example, suppose the problem had a size of N = 50. This requires 7548
steps, which to a human seems difficult, but effortless for a computer.
2) Non-Deterministic Polynomial Time Problems (NP Hard)
NP Hard problems are a batch of problems which even computationally are demanding.
Their completion steps are an exponential function of the input steps, rather than
(12)

Design of Binary Fringe Patterns for High Speed 3-D Vision: 6: Optimisation of Binary Patterns
Page | 29
polynomial. Now using the same problem size N = 50 as before and a simple exponential
function for solving steps, S:
𝐸𝑥𝑎𝑚𝑝𝑙𝑒: 𝑆 = 2 𝑁
250
= 1.13𝑥1015
It is of no debate that NP hard problems take a substantial amount of more steps to
completely solve compared to polynomial time ones.
In the case of binary images, suppose a frame of β = 12 and δ = 12. There are 144 pixels
present (N = 144) and assuming there are N/2 black pixels and N/2 white pixels. The number
of combinations of pixels would be
𝑁!
(
𝑁
2
) ! (
𝑁
2
) !
=
144!
72! 72!
= 1.48𝑥1042
which is the Combinatorial Function [26], adapted for the case of 12x12 binary image.
Even for this small binary pattern size, the amount of combinations that would have to be
processed is very unrealistic and would take a considerable amount of time to solve. Also
note that this is only for a 12x12 image, when the patterns in Section 5 had β values up to 96.
Because of this, optimisation algorithms are crucial to finding an optimal arrangement as
solving the problem computationally from scratch is practically impossible. This highlights the
need for the best image generation as in Section 5, as a low initial RMSE prior to optimisation
provides the best chance for finding a minimal solution.
(13)
(14)

Page | 30
6.2 – Mixed Linear Integer Programming (MLIP)
This is the first optimisation method used for this paper. Due to constraints with programming
it and the general nature of its use, the results are not really valid and useful for the final
conclusions in Section 6.6. For informative reasons and completeness, the method has still
been included for this section, but it is not necessary to fully understand it, therefore the
reader may skip to Section 6.3 should they wish to for a first reading.
6.2.1 – Overview of MLIP
The first of these optimisation algorithms works on trying to converge the pixels’ intensity in
an array to best match a sinusoid’s intensity. When convolution occurs, each pixel along with
a number of its surrounding pixels undergo having their average taken, in the aim that the
central pixel’s intensity will match that of a sinusoid at each (x,y) value, as shown in Figure 23.
1 0 1
0 0 1
0 1 0
Δ
a)
b)
Figure 23:
a) Sine wave formed of the average of intensities of pixels. Example: Average of I11 to I33 should provide close to an ideal sinusoid
intensity at location 122
b) The example patch of 111 to I33, comprised of binary intensities and of dimensions Δ. The average of this grid should approximately
equate to an ideal sinusoid intensity of the middle cell.

Page | 31
Now, the initial binary image generated should do a relatively good job of approximating a
sinusoid by taking the average of central and surrounding pixels for the whole array.
However, there is room to optimise this further and to find the best combination of pixels.
This is done in this case through the linear programming algorithm [27], which maximises or
minimises a cost function through a combination of set variables. Mixed Linear Integer
Programming (MLIP) ensures that the variables can only be integer values.
In the case of optimising our binary pattern, the intensities I11, … IΔΔ are programmed so that
the average intensity of the patch is within the ideal sinusoid intensity at that location
plus/minus a tolerance level, e. To ensure the intensity variables are binary, integer linear
programming is implemented under the limits the variables can only be 0 or 1.
With these limits, the objective of the optimisation at each pixel is:
𝐼𝐼𝑑𝑒𝑎𝑙 − 𝑒 < 𝐼∆ < 𝐼𝐼𝑑𝑒𝑎𝑙 + 𝑒
Where IΔ is the averaged patch intensity, IIdeal is the sinusoidal intensity and e is the tolerance.
Matlab’s ‘intlinprog’ function can solve this for every point (x,y) on the binary image such
that:
𝑀𝑖𝑛𝑖𝑚𝑖𝑠𝑒:
1
∆2
(𝐼11 + 𝐼12+. . . 𝐼∆∆) 𝑆𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 {
𝐼∆ > 𝐼𝐼𝑑𝑒𝑎𝑙 − 𝑒
𝐼∆ < 𝐼𝐼𝑑𝑒𝑎𝑙 + 𝑒
𝐼 𝑥,𝑦 = 1, 0
}
The above is the setup of MLIP to be programmed into MATLAB. The left equation is the
average of intensities in the patch of height/width Δ. The right-hand side limits the
implementation of Eqn. 15 and provides the binary limits.
After each pixel in the array is processed, the procedure is repeated in an iterative manner
until a converged solution is found. It finds the right combination of 1s and 0s to meet the
limit criteria.
(15)
(16)

Page | 32
6.2.2 – Issues Associated with MLIP
There are several issues with this method, however. At no point does this algorithm consider,
kernel blurring, phase shifting, unwrapping, calculating phase or RMSE. The Matlab function
can only process simple linear cost functions, such as the difference in intensity average.
Because of this, the phase of the blurred image can only be determined once MLIP
optimisation is complete and there is no guarantee that the RMSE will even be improved as a
result. Quite commonly when carrying this method out, the phase error actually increased
upon completion.
Also, the averages of the batch are supposed to reproduce the blurring effect previously done
by convolution. Due to the limitations of the function, convolution could not be implemented
as part of the MLIP equation, hence why the patch average was used instead. Furthermore,
the usual patch dimensions could only reach around Δ = 5 before the computational time rises
considerably. To truly replicate the blurring in previous sections, the MLIP patches would have
to be of size 9x9 up to 31x31 as used in Section 5, which is way beyond the computational
capability of the MLIP function.
A final key issue with this algorithm is that it has no way of determining where to place the
optimal cell values in the patch. For example, if it worked out that in a 3x3 patch, 5 should be
black and 4 white, the computer has no way to tell where these pixels should be located in
the patch. It distributes them randomly and therefore this will have impacts on the quality of
neighbouring patches.
6.2.3 – Summary
Whilst this method does come with several issues, it did improve the RMSE in some cases. In
the future, if the MLIP function evolves and becomes able to process RMSE as a cost function,
then it should definitely be considered for re-evaluation. It is quite clear at this point in time
however, it is quite impractical to use compared to other optimisation methods.
Nevertheless, for informative purposes, the results of the MLIP algorithm can still be found in
Section 6.6.

Page | 33
6.3 – Pixel Mutation Method
The second optimisation method is one taken from J. Dai et al [6] which utilises the phase
difference at each point in the image. When a binary image is generated, some pixels will
have a lower phase error compared to others. Dai’s ‘pixel mutation’ method takes a pre-
existing image, commences blurring and then plots a phase error map of the blurred image,
such that:
𝜙 𝐸𝑟𝑟𝑜𝑟(𝑥, 𝑦) = |𝜙𝐼𝑑𝑒𝑎𝑙(𝑥, 𝑦) − 𝜙 𝐵𝑙𝑢𝑟𝑟𝑒𝑑(𝑥, 𝑦)|
Then, by setting a tolerance level, e, a binary error map is formed. Starting with an empty
array, a 1 is plotted at coordinate (x,y) if the absolute phase error at position (x,y) is greater
than e. In doing so, you have a map which highlights where the key RMSE contributors are
located, called the ‘error pixels’.
The next step of the process is to individually change the error pixel’s intensity, (1 ↔ 0) and
then rework the whole phase map. If the new RMSE is lower than the previous, then this is
treated as a ‘good’ change and this new binary image is accepted. This process evaluates the
change in every error pixel, but because changing one pixel will impact neighbouring ones,
the whole algorithm has repeated iteratively. Figure 24 flowcharts this process and Figure 25
shows an example of such.
This method, unlike the MLIP of Section 6.2, will definitely improve the image as the RMSE is
the cost function of this algorithm.
(17)

Page | 34
Start:
Binary Image from Generation
Algorithm
Calculate Phase Map
From Eqn. 17
Is |ΔΦ| > e at pixel
(x,y)?
Change pixel (x,y) from 1
to 0 or 0 to 1
Rework phase error
Is RMSE improved?
Accept changed pixel and
treat new binary image
as original
Travel across error map
using coordinates (x,y)
n iterations reached?
End
Yes
Yes
Yes
No
No
No
Has whole image been
processed?
No
Yes
Figure 24:
Flowchart of the Pixel
Mutation method

Page | 35
Figure 25: Example of the Pixel Mutation Method in use
Top Left: Original binary image at start of algorithm (Randomised Bits, β = 96)
Top Right: Gaussian Blurred Image
Middle Left: Error Map, based on Eqn. 17
Middle Right: Binary Error Map. If change in error is greater than tolerance e, a 1 is plotted and this point is processed as in Figure 24.
Bottom Left: Post – Optimisation Image
Bottom Right: Gaussian Blurred Optimised Image

Page | 36
Because the tolerance level in this algorithm can be pre-decided by the user, one can set how
low they roughly want their RMSE to go, however care must be taken when setting this
tolerance. If it is made too low, a solution for this criteria may not exist and the computer will
spend an excessive amount of time searching for an impossible combination.
Overall the Pixel Mutation method provides promising results, especially compared to that of
MLIP. Section 6.6 contains the results for this method.

Page | 37
6.4 – The Theory of Local / Global Minima
Before discussing the third and final method of optimisation, it is important to understand
local and global minima. In optimisation algorithms, the overall aim is to find what is known as
‘the global minimum’, which is the absolute best combination of variables possible. In this
case, it would be the perfect arrangement of on/off pixels to provide the lowest RMSE
achievable. With the previous algorithm, the general process has always been to repeatedly
accept the best value, which may initially make someone assume will eventually take the
RMSE down to the lowest possible value.
This however is not the case, using Figure 26 as an example. If an algorithm starts at position
A then randomly keeps on accepting the best values it comes across, the lowest point it can
get to the local minimum at point B. Once it is here, the solution becomes ‘stuck’ at this
position because it cannot accept any better values, but it is clear that the best value to reach
overall is at C, the global minimum. This raises the importance of being able to implement
commands in an optimisation technique that can ‘climb’ out of a local minimum and exhaust
every avenue of solutions so it can find the global minimum, rather than only the local.
A
B
C
Figure 26: Example of how one can get ‘stuck’ in a local minimum from A to B, but
really the overall minima is in fact at C

Page | 38
Of course, the case in this paper is a lot more complex than that of the above figure, as the
number of combinations of pixels is very high, as discussed in Section 6.1. It also highlighted
the importance of having a good initial generation algorithm, as it means one is closer to the
global minimum and has a greater chance of reaching such. The following technique takes
advantage of the points mentioned above.

Page | 39
6.5 – Simulated Annealing
The final method for optimisation in this paper combines a new image refinement algorithm
alongside a common probability technique to find global minima. The first part of this section
explains the classical Simulated Annealing algorithm, in which how the global minimum will
be found (Section 6.5.1). Following this is the new binary refinement method to be used,
similar to Pixel Mutation, which will then be combined with the annealing principle. After
highlighting the issues with using the classical Simulated Annealing process, newly proposed
changes made to the algorithm are discussed, leading to a new technique used in Section
6.5.4. Ultimately this provides a new optimisation process which presents the most
encouraging results yet.
6.5.1 – Annealing Based on Probability
Simulated Annealing is an optimisation technique that classically works on a probability basis
[28], that after a certain amount of iterations a worse value is accepted, with the intent on
‘climbing out’ of the local minima. Its name is derived from the materials annealing process, a
popular manufacturing technique. In real life annealing, an object is heated to an extreme
temperature, sustained and then cooled down, aiming to strengthen its material properties.
In this case with the pixel swap method, if Simulated Annealing is applied to it, the algorithm
will extensively keep accepting better values, but then eventually it will accept a worse swap
of pixels which have a higher RMSE than the previous. Temporarily this will increase the
overall phase error, but after a while it should converge to a value lower than the one prior to
accepting the worse combination. If this was carried out repeatedly, the global minimum
could be achieved, providing the lowest RMSE possible.
The difficult part of applying simulated annealing is implementing the correct ‘cooling
schedule’, which is the probability at which worse values are accepted. Ideally the chance of
worse acceptance should be kept very low so it does not interfere with the pixel swapping too
much. The probability of worse acceptance follows the equation:
𝑃 𝑊𝑜𝑟𝑠𝑒 = 𝐴𝑐𝑐𝑒𝑝𝑡𝑒𝑑 𝑖𝑓: 𝑒−
Δ𝐶𝐹
𝑇 < 𝑅𝑎𝑛𝑑𝑜𝑚(0,1) (18)

Page | 40
where P is the probability of accepting a worse value [29], CF is the cost function (RMSE in our
case, magnitude of ≈ 0.01) and T is the ‘temperature’. Random value is between 0 & 1.
The cooling schedule is the trend ‘T’ follows in Eqn. 18 and can be experimented with. At the
start of Simulated Annealing, it is ideal to allow the system to run for a period of time
unaltered so that the process can generally converge near enough to the global minimum. If
Eqn. 18 starts accepting worse values as soon as possible, the RMSE will end up spiralling out
of control and will be no chance of finding a minimum. Because of this, at early values of
iterations, P should be kept very low (practically 0) then gradually increase. At latter values of
iterations when local minima convergence is high, P should be approximately 1x10-4. This may
still seem really small, but accepting the worse value should be a last resort, assuming the
local minimum has truly been met. Also, these values suggest P is accepted once every couple
thousand cycles, which isn’t too time consuming for a computer processing this algorithm.
Figure 27 shows a variety of cooling schedules that could be considered for this application:
Figure 27: Examples of different cooling schedules
Any cooling schedule can be implemented into a
simulated annealing problem. These work so the
chances of worse acceptance increase the nearer
one is to a local minimum.

Page | 41
6.5.2 – The Pixel Swapping Method
Similar to the Pixel Mutation method of Section 6.3, a newly proposed process named ‘Pixel
Swapping’ has been implemented on binary images with promising results. It works by taking
the pattern and randomly swapping two pixels of opposing intensities, then working out the
phase of the newly formed image and accepting it if it is regarded as a good change, i.e. if the
RMSE is decreased.
A limit worth noting with this method is that the optimised image is constrained to the
number of certain pixels it starts with. There are no more black/white pixels present after
optimisation compared to before, due to the fact pixels are only being ‘swapped’ rather than
having their intensities change.
Eventually after thousands of iterations of swapping and accepting pixels, the local minima
will inevitably be met, such as in Figure 26. As explained in Section 6.4, it is ideal to find a way
to get out of this minimum and then resume converging solutions to find the global minimum.
This is where ‘Simulated Annealing’ comes into play. Combining this swapping of pixels with
the annealing principle, it allows the global minimum to be sought after.
Figure 28: The Pixel Swapping Method example

Page | 42
6.5.3 – Issue with Probability Based Annealing with Pixel Swapping
It was found that when using a probability based technique for optimisation, there were
numerous times where the RMSE was converging to a promisingly low value, but due to
random chance with Eqn. 18, it would rise back up again due to acceptance of a worse swap.
When this happens, potentially a good avenue of solutions is lost and may not be reached
again.
Take Figure 29 for example, the solution trend appears to be converging to a newest low
RMSE value, but due to the probabilistic nature of Eqn. 18, a worse value becomes accepted
before this solution path is fully
investigated. After this, the
algorithm converges to a new
low RMSE, 𝜙 2, but it would
have been better to remain on
path to find the lower previous
potential RMSE, 𝜙 1.
This is one of the key issues
found with using probability in
optimisation methods. Where
the smallest change in pixel
swaps can have large impacts
on the phase error, avenues of
solutions should be fully
exhausted before attempting
to solve a new one.
This leads to the reason why it has been decided to alter the simulated annealing technique
to work off a convergence basis, not a probability one.
6.5.4 – Annealing Based on Convergence
As explained previously, it is unwanted for the Simulated Annealing algorithm to start
rejecting paths of solutions where they could potentially lead to a global minimum. So,
Figure 29: Demonstration of worse acceptance impact
Originally, the solution would converge down to 𝜙 1, but because of a worse
swap being accepted, it will never reach this value and end up converging to
a higher RMSE 𝜙 2

Page | 43
instead of accepting worse values due to probability, this new technique works on examining
the change in cost function. If the phase RMSE converges and goes under an amount of N
iterations (say 25,000) and no new minimum is found, then it can be safe to assume that this
path of solution is ‘exhausted’ and the phase can get no lower. Then at this point, it is logical
to accept a worse value to climb out of this local minimum and then process can resume.
In doing this, it is almost impossible to miss on potential low RMSE values and will significantly
increase the chance of reaching global minima. Figure 30 graphically demonstrates annealing
based on convergence.
Because this change to the technique involves completely processing every solution path
encountered, it is quite time consuming as a result and this should be considered.
Figure 30: Example of convergence Simulated Annealing
See how trend has been completely exhausted before starting a new path.
Worse value acceptance occurs after a set number N of iterations occur
without a change

Page | 44
An example of this technique is found in Figure 31, where it a result taken from Section 6.6.1:
The full results for this form of annealing can be found in Section 6.6.
6.5.5 – Using a Three Frame Annealing Technique
The final change made to this implementation of simulated annealing took advantage of the
DLP projector’s 3 frame nature. Throughout this paper, it has been highlighted that the key
aim is to have a projected image which has an intensity distribution that replicates that of a
sinusoid. Looking carefully at Eqn. 8 and Sections 5 & 6, it is stated that our cost function (and
more importantly, what the computer is looking for) is actually the phase of the projected
images, not necessarily how well images replicate a sinusoid. Of course, having a sinusoidal
intensity binary image will most likely provide a close to ideal phase, but the process is not
constrained to this.
Figure 31: Example of single frame Simulated Annealing, taken from Section 6.6.1 (β = 96)
Note after major peaks in error (worse value acceptance), the RMSE converges to new
lowest values

Page | 45
DMD Discovery boards available contain on-board flash memory used for storing and
projecting three images onto objects [11], which have been used to project the same binary
patterns, but each one is shifted by 2π/3. However, there is no restraint that forces the three
images to be identical to each-other. If the three shifted frames look completely different, but
provide a better RMSE of phase, there is no constraint disallowing this, making it acceptable
to use this sequence. This thinking was applied when making the final change to the
Simulated Annealing algorithm, in that rather than optimising one image and shifting it, three
images would simultaneously be processed and the best combination of such will be used.
Figure 32 shows the implementation of this. At the start of the process, all three images will
be identical and shifted by 2π/3 as usual. Frame 1 is then processed and an RMSE is found,
then undergoes pixel swap and a second RMSE is found. Whichever of these is lowest will
determine if the swap is kept or not. Then this image is passed on and the same process is
applied to Frame 2, then Frame 3. In total as shown in Figure 32, this provides 8 different
combinations of a mix of swapped/un-swapped binary images. Once Frame 3 is complete and
the cycle is finished, the three confirmed images are sent back to the start for the next
iteration.
Figure 32: 3 Frame Simulated Annealing diagram
Ixy is the binary image where x is the image number (1,2 or 3) and y if it is kept changed or not (1 = swap not
accepted, 2 = swap accepted)
I11
I11
I12
I11,21
I11,22
I11,21,31
I11,21,32
I11,22,31
I11,22,32
I12,21,31
I12,21,32
I12,22,31
I12,22,32
I12,21
I12,22
I.e. completely unchanged
I.e. All 3 images changed
Binary Image 1 Binary Image 2 Binary Image 3

Page | 46
Combining this with the convergence annealing of Section 6.5.4, once all three images stay at
a minimum RMSE with no sign of improvement as per Figure 30, the three pixel swapped
images (I12,22,32 in Figure 32) are accepted for the next iteration, regardless if they improve
RMSE or not.
In applying a 3-Frame technique, it gives the algorithm 3x more flexibility to find a new global
minimum, which theoretically should be lower than using an identical 2π/3 shift image. Using
this method, however, combined with convergence annealing, is considerably more
computationally demanding and sufficient time should be dedicated to the computer for
optimising with this method.

Page | 47
6.6 – Results of Binary Optimisation
The following is a discussion of the optimisation results of the methods explained throughout
Section 6. Because ultimately if β and δ were kept constant, regardless of what image
generation technique was applied, the overall global minimum should be closely similar and
would provide like results. Due to this, only the Randomised Bits generated images are used
for this section as they provided the best initial results and will most likely converge to global
minima quicker.
Secondly, to provide a direct comparison with pre-existing papers, only Gaussian blurring will
be used when optimising the images, with a filter size of D = 13 and σ = 13/3.
6.6.1 – Overall Results – Discussion
On the few following pages, the complete results of the optimisation techniques (Figure 34)
can be found, alongside the original unaltered versions (as in Section 5), all as a function of β.
Certain techniques outweigh others in all aspects, with some reaching as low as a 0.0024 RMS
error (3 Frame SA, β = 72).
Mixed Linear Integer Programming
The Mixed Linear Integer Programming method clearly displays the worst results, only
improving the RMSE on one β value and actually increasing the original error as the period rises.
As explained in Section 6.2.2, this method does come with many drawbacks that have evidently
had a large impact on results. It is confident to assume that the main reason for this high error
trend is the fact the MLIP method is the only one in this paper with a cost function that isn’t
RMSE. It is working to approximate a sinusoid’s average intensity without considering phase,
which therefore does not guarantee a better RMSE. Another smaller reason for poor results
include the lack of being able to use a high Δ value due to computational constraints.
Overall I wouldn’t consider the MLIP method at this stage as an optimisation method as it has
done the opposite of what was intended of it. As discussed in Section 6.2.3 however, if this
method could implement RMSE as its cost function in the future, then it is definitely a
potential avenue for re-investigation.

Page | 48
Pixel Mutation
This method provided much better results compared to MLIP, mainly utilising the fact the cost
function is RMSE in this case. It is curious to see how the error map tolerance, e was set to be
0.005, but all the final solutions are over double this. This will be due to the fact of changing
one pixel’s intensity will impact the surrounding pixels, highlighting the need for this method
to undergo an iterative process rather than just implementing it once. Even at a time
consuming 15 iterations, this phenomenon still occurs and the RMS errors are considerably
higher than what was set.
Nevertheless, this method still provides encouraging results in reducing the phase errors. It
seems to provide a near constant RMSE at all values of β, better than the MLIP method where
the errors escalate. For future work on this method, one could set the error tolerance
considerably lower than 0.005 and increase the iteration number to much higher values, say
50 – 100, of course at the expense of high computational times. This could allow the user to
explore the full use of this method.
Single Frame Pixel Swap + Simulated Annealing
This method’s results provide a much lower error trend compared to the previous, for
example taking an original RMSE of 0.08 to 0.004 rad at β = 24 pixels. It takes good advantage
of the annealing principle, as during the process once local minima were reached, the
solutions were ‘relaxed’ and a worse match was accepted, then allowing the RMSE to lower to
unprecedented values. Again, this method seems to work irrespective to the period size β,
which allows a user to confidently implement this method in their DLP projector at any fringe
size.

Page | 49
3 Frame Pixel Swap + Simulated Annealing
Unsurprisingly this method proved to be the best overall, with the lowest errors at all values
of β. Utilising the flexibility of not having the constraint of identical images, the cost function
of the phase could reach as close to of that of a real sinusoid. Overall with annealing, there is
still no guarantee that these values found were the absolute global minimums, therefore it is
always worth doing several attempts of the process at each β value.
In general, it is interesting to see how well some methods do compared to others, with there
being over a magnitude difference in some cases. The MLIP proved to be the worst, especially
due to the number of flaws and drawbacks it came with that made optimisation impractical.
The Simulated Annealing method is the best as it is the only one that allows an algorithm to
search for a global minimum, and the results of Figure 34 show how important this turned out
to be.
Figure 33: Example of a blurred image’s sinusoid intensity vs an ideal (Simulated Annealing).
Taken from the results of Figure 34: β = 48, RMSE = 0.0047 rad.

Page | 50
Figure 34: Results of Section 6’s optimisation methods
Top Left: MLIP, Top Right: Pixel Mutation, Middle Left: Single frame Simulated Annealing, Middle Right: 3 Frame Simulated
Annealing. Bottom: Combined single & 3 frame annealing
Gaussian Blur: σ = 13/3, size 13x13

Page | 51
6.6.2 – Comparison to Literature
Figure 35 below shows a direct comparison to between a previous recent method by X Li et al
[20] in 2016 and the best method in this paper (i.e. 3 Frame Simulated Annealing). From this it
shows an improvement by roughly 0.002 rad, showing an approximate 40% improvement
over his results. This was done using the exact same parameters of filter type, size, deviation
etc. (those used in Section 6.5).
This provides confidence in the results collected in that they are effective and could be
successfully implemented into a DLP projection system for experimental use.
Figure 35: Comparison to the Simulated Annealing Method in this paper Vs X Li’s
proposed method

Page | 52
7 – Realistic Considerations
Throughout this paper and others, the DMD projection system has been simplified for
simulation purposes and for simplicity. These simplifications will have an impact on
experimental results, as the RMS phase error may be considerably higher than what was
predicted in models and algorithms such as those in this paper. This following section looks
at a key factor of DMDs which most papers seem to have neglected in their models. The aim
is to see if this factor can be included in a simulation algorithm and examine the impacts it
will have on the RMSE of phase.
7.1 – Considering Inter-Pixel Gap
DMDs as explained in Section 2 and Section 3 are formed of an array of tiny mirrors which
reflect light to project images. These mirrors are closely spaced together, but not
infinitesimally compact. They will all contain some degree of gap between them, which will
contain regions of no projected intensity, as shown in Figure 36. Most papers neglect this
when carrying out simulations, with only some considering the impacts it has [30]. With
sudden discontinuities in intensity as a result of pixel gaps, the convolved intensity can face
immediate drops which may require a stronger blur to overcome.
The aim of this section is to find a minimum blurring level to be used which will overcome
the pixel gap issues. For such analysis, a pixel fill factor, γ of 92% percent will be used as this
is a set value for a majority of Texas Instrument DMDs [31].
Figure 36: Left: simplified DMD arrangement with no pixel gap
Right: realistic DMD arrangement with gap included
γ = 100% γ = 92%

Design of Binary Fringe Patterns for High Speed 3-D Vision: 7: Realistic Considerations
Page | 53
To visualise DMD pixel gap and implement it in a simulation, each pixel will be represented
in the form of Figure 37. The ‘pixel’, P will be the whole area of a single pixel including the
surrounding gaps, and this will contain a number, n of theoretical ‘micro-pixels’, Pm which
are the region of this area where the pixel is actually located, governed by the fill factor.
This therefore leaves a gap ω between neighbouring pixels.
Using this setup, the following
relationship between P, mP, ω and γ is:
(𝑛𝑃𝑚)2
= 𝛾(𝑃)2
(𝑛𝑃𝑚) = (𝑃 − 𝜔)
(𝑃 − 𝜔)2
= 𝛾(𝑃)2
(1 − √ 𝛾)𝑃 = 𝜔
So, in using Eqn. 19.4, if γ = 0.92 as
specified by Texas Instruments [31]:
ω ≈ 0.04P
Using the above result, a value of P = 50 is used, so that the pixel is comprised of 50
micropixels and the total gap ω is a round number of 2, all simple for calculations and
simulations.
The first part of this analysis will be done by taking a perfect greyscale sinusoid which will be
subjected to pixel gaps. Starting off with a weak blur, the phase is repeatedly calculated as
the blur diameter gradually increases. This will be able to show the relationship between
RMSE and blur size, allowing a user to know what the minimum amount of blurring should
be implemented in a system to overcome gaps. This will be followed by a brief discussion on
how these effect binary cases and then finally the RMSE of the 3 Frame Annealing results
will be re-calculated with included gaps.
P
P
nPm
Figure 37: How pixel gap will be represented in the
forthcoming simulations. This is a single pixel.
ω/2
(19.1)
(19.2)
(19.3)
(19.4)

Page | 54
7.2 – Inter-Pixel Gap Simulations
7.2.1 – Greyscale Sinusoid with Gaps
The following test was run on a greyscale sinusoid, originally with no gaps, then with a fill
area of 92%.
In the cases of all sections leading up to this, the optimised binary images have been
represented as a matrix of dimension δxβ, with each cell value representing one pixel. In
reality for DMDs, the pixel will have a complete area in which its designated intensity is
uniform. This results in a phase plot like that of Figure 38, where each pixel results in a step
looking function with constant phase.
Figure 39 demonstrates how increased blurring works to eliminate the presence of gaps and
allows for a smooth phase line to be formed as intended.
Figure 38:
Left: Greyscale sinusoid with and without gaps.
Top: Resulting phase plot of gapped case
The staircase plot is a result of the DMD having a uniform
intensity on each pixel. Stronger blurring will eventually
compress this stair plot into a straight line.

Page | 55
Figure 39: A gapped greyscale image with increasing blur strength & corresponding phase
Notice how the images become smoother as blur strength increases

Page | 56
Figure 40 above is the final analysis of the greyscale case. As expected, it does suggest a
minimal degree of blurring is to be implemented to a system, which seems crucial at smaller
diameters of blurring. After the first steep initial drop, the RMSE begins to converge to small
value. The two plots appear identical in their trends, with the gapped case slightly more
shifted to the right, most assumedly being the requirement of slightly more blur to
overcome these gaps.
This demonstrates that even when not working in a binary pixel case, blurring is still
required in a system in order to overcome gaps. For binary cases, it may be wise to combine
the two amounts of blurring worked out, i.e. the blurring to overcome gaps (Figure 40) and
the blurring to form a sinusoidal approximation (Sections 4,5,6,).
Figure 40: RMSE Vs Gaussian blur diameter for gap/no-gap case

Page | 57
7.2.2 – Inter-Pixel Gap Effects on Simulated Annealing Results
This final sub-section on pixel gap takes the 3 Frame Annealing results from Section 6.6.1
and re-calculates the phase of them, but with gaps included. This can demonstrate the
magnitude of difference by considering this factor.
Examining Figure 41 (following page), one can witness the impacts of adding gaps to a
system. It appears to increase the phase errors at each value of β, roughly by about 0.002 –
0.008 rad. This supports the greyscale results of Figure 40 as the difference between
gapped/un-gapped in that case was also by a similar amount.
This does open future potential investigation, being that someone could attempt to
implement an optimisation algorithm such as Simulated Annealing, but with gaps
throughout. The main drawback of this is that the un-gapped algorithm already is quite
computationally demanding, so adding gaps would increase the number of process by x50
(i.e. P = 50Pm). Nevertheless, as computational capability grows, this gapped optimisation
technique could be carried out.
7.3 – Discussion of Realistic Considerations
This whole section should provide an insight into some of the more complex factors
associated with DMDs and their impacts on simplified results. Pixel gap is only one of these
factors and already it demonstrates a significant effect. Undoubetdly there will be more
elements in projectors that will increase the errors of phase compared to those worked out
in the likes of Section 6. Because of this, the more factors included in simulations will
provide the best representation of what one could expect to see in an experimental
environment.

Page | 58
Figure 41: impacts of adding pixel gaps to previous results
Top: Example of a gapped/un-gapped region of a binary image
Middle: Phase lines of corresponding images. Close inspection can show more deviation around ideal phase due to gaps
Bottom: RMSE vs β with and without gaps, note rise in phase error.
(Gaussian blur D = 13, σ = 13/3)

Page | 59
8 – Summary & Conclusions
8.1 – Closing Summary & Conclusions
Throughout this paper, the design of binary patterns has been extensively investigated.
Starting with the theory of how structured light works, the chronology of 3-D acquisition
techniques has been discussed, leading to the need for binary imaging in the current day.
Numerous algorithms of generating and optimising binary arrangements have been
explained and thoroughly tested to confidently identify which of these provide the best
possible sinusoidal intensity.
By examining the phase plots of the unwrapped binary frames, sufficient numerical data has
allowed the clarification of the best and worst binary imaging techniques. The Randomised
Bits method of Section 5.1 proved to be the leading method in forming a frame from
nothing, outweighing other methods suggested by those found in literature.
For optimisation, the combination of pixel swapping, using a 3-frame basis and modifying
the common Simulated Annealing principle (Section 6.5) has provided a leading technique,
whose results are unrivalled by the other methods in this paper and those found in the
literature. Achieving constant RMS phase errors of around 0.003 rads appears
unprecedented for this type of simulation. It has achieved an RMS error improvement by
>40% compared to previous state of the art results (Figure 35), showing a clear success in
the Simulated Annealing algorithm.
After the simplified processes were carried out, discussion was made on the realistic factors
associated with DLP projectors. A key highlighted element was the characteristic of pixel
gaps in DMDs, a detail not commonly included in the working out of phase errors in
simulations. After incorporating this into a new computer test, plots were made to show the
impact pixel gaps has on greyscale images and the leading results of the Simulated
Annealing. They demonstrated a considerable rise in phase errors and highlight the need for
including characteristics of projectors that are often disregarded in simple simulations.
Using the methods of this paper, refined, optimal binary patterns can be obtained by any
user for use in a 3-D projection-based profilometry with high accuracy results. With DMDs
able to cycle between images at 30kHz [32], using these generated fringes can offer a wide
variety of opportunities and applications.

Design of Binary Fringe Patterns for High Speed 3-D Vision: 8: Summary & Conclusions
Page | 60
8.2 – Potential for Future Work and Further Investigation
Whilst there is confidence in the results presented in this paper, the methods used are far
from conclusive and there are still certainly more areas in this field that require
investigation. Either due to time constraints or unavailability of resources, the following
research areas were unable to be explored and carry potential for future work:
1) Experimental Procedure for the Results in This Paper
All the research and data for this paper has been carried out in a simulation basis on
MATLAB. Whilst the results of which are still valid, it would have been interesting to see how
well they perform when programmed into a DMD board and projected to record some real
3-D data. Especially for the simulated annealing results which in a simulation basis display
the leading results, it is curious to see if these values are closely replicated in a real-life
basis. This would undoubtedly open more avenues for investigation to be carried out and
shows itself as the clear next step for future work.
2) Full Extensive Optimisation Run-Through
A maybe less important potential for future work, but still a valid one is the collection of
data for the results in Section 6. The optimisation algorithms presented in this paper have all
been carried out on a single personal computer with an Intel i7 processor. Run times for
Pixel Mutation and Simulated Annealing have ranged between 2-8 hours for data points
valid enough to present in Section 6.6. Whilst still quite time consuming, this does not
guarantee that the absolute global minimum for these scenarios have been met. If one has
access to a lab with multiple, more powerful computers designed for tasks such as this, it is
possible to simultaneously re-run these optimisation algorithms at much longer run times in
the aim to providing the best results possible.

Page | 61
9 – Acknowledgements
I would like to thank my project supervisor Professor Jonathan Huntley for his continued
support throughout the past 8 months in developing this research, always offering support
and helping me almost immediately when I require such. He has taken my knowledge of this
field from practically nothing to the stage it is today, alongside making me more than
comfortable in using MATLAB, which will without a doubt benefit me in my career after
university.
I thank Dr Pablo Ruiz for his feedback in this project and his encouragement of my progress.
I would finally like to owe the success of my research and development as an engineer to
the Wolfson School of Mechanical, Manufacturing & Electrical Engineering, Loughborough
University.

Design of binary fringe patterns for high speed 3 d vision

Design of binary fringe patterns for high speed 3 d vision

Recommended

Recommended

More Related Content

What's hot

What's hot (18)

Similar to Design of binary fringe patterns for high speed 3 d vision

Similar to Design of binary fringe patterns for high speed 3 d vision (20)

Recently uploaded

Recently uploaded (20)

Design of binary fringe patterns for high speed 3 d vision