project report final

Birla Institute of Technology and Science, Pilani
1
A Project Report
On
Coded Target Detection and 3D Coordinate
Points Generation
Name of the Students ID
Sankhya Chakravarty 2012A7PS014P
Aishwarya Gupta 2012A2PS241P
Kunjesh Agashiwala 2012B5A3663P
Ishan Kumar 2012AAPS246H
Vishal Kejriwal 2012A2PS366P
Shubham Singh 2012B3A7466P
Divija Gogineni 2012AAPS003H
Jaiwant Rawat 2012B4A7714P
Prepared in partial fulfillment of the
Practice School – 1
Course No. – BITS F221
At
Indian Institute of Remote Sensing, Dehradun
A Practice School – 1 station of
BIRLA INSTITUTE OF TECHNOLOGY AND SCIENCE, PILANI
23rd
May to 17th
July, 2014

2
A Project Report
On
Coded Target Detection and 3D Coordinate
Points Generation
Name of the Students ID Discipline
Sankhya Chakravarty 2012A7PS014P (B.E.) Computer Science
Aishwarya Gupta 2012A2PS241P (B.E.) Civil
Kunjesh Agashiwala 2012B5A3663P (M.Sc.) Physics+
(B.E.)Electrical and Electronics
Ishan Kumar 2012AAPS246H (B.E.) Electronics
&Communication
Vishal Kejriwal 2012A2PS366P (B.E.) Civil
Shubham Singh 2012B3A7466P (M.Sc.) Economics + (B.E.)
Computer Science
Divija Gogineni 2012AAPS003H (B.E.) Electronics
&Communication
Jaiwant Rawat 2012B4A7714P (M.Sc.) Maths + (B.E.)
Computer Science
Prepared in partial fulfillment of the
Practice School – 1
Course No. – BITS F221
At
Indian Institute of Remote Sensing, Dehradun
A Practice School – 1 station of
BIRLA INSTITUTE OF TECHNOLOGY AND SCIENCE, PILANI
23rd
May to 17th
July, 2014

3
Declaration
We hereby declare that the work recorded in this project report entitled “Coded Target
Detection and 3D Coordinate Points Generation” in partial fulfillment for the Practice School –
1 of Birla Institute of Technology and Science, Pilani is a faithful and bonafide project work
carried out at “Indian Institute Of Remote Sensing Dehradun” under the supervision and
guidance of Ms. Shefali Agarwal, Mr. Raghavendra S, Ms. Poonam S Tiwari and Ms. Hina
Pande.
The results of this investigation reported in this project have so far not been reported for
any other Degree / Diploma or any other Technical forum.
The assistance and help received during the course of the investigation have been duly
acknowledged.
Name of the Students ID
Sankhya Chakravarty 2012A7PS014P
Aishwarya Gupta 2012A2PS241P
Kunjesh Agashiwala 2012B5A3663P
Ishan Kumar 2012AAPS246H
Vishal Kejriwal 2012A2PS366P
Shubham Singh 2012B3A7466P
Divija Gogineni 2012AAPS003H
Jaiwant Rawat 2012B4A7714P

4
Acknowledgement
We take this opportunity to express our gratitude towards everyone who has made it
possible for us to work on this project.
Sincere gratitude and thanks are expressed to Ms. Shefali Agarwal, Mr. Raghavendra S,
Ms. Poonam S Tiwari and Ms. Hina Pande of IIRS Dehradun for sharing their valuable
knowledge about the subject and helping us with their ideas and resource.
We would like to express our thanks to Dr. Pradipta Chattopadhyay, PS-1 Faculty
Coordinator and Mr. Vasant Keshav of BITS Pilani for making the facilities available to us
whenever needed and also giving us encouragement during the course of this project.
We would also like to extend our heartfelt gratitude towards BITS Pilani for allowing us
to proceed with the project and lastly our heartiest thanks to all our friends who helped us in this
project and gave us inspiration and confidence.

5
Contents
1. Abstract 7
2. Introduction 7
2.1. Theory and Overview 7
2.2. Documents and Research Papers 9
2.3. Organization of the report 10
3. Problem Statement and Analysis 11
3.1. Problem Statement 11
3.2. Detailed Problem Analysis 11
3.3. Solution Strategies 11
3.4. Hardware and Software Requirements 11
4. Algorithm Design 12
4.1. Entire Flow Chart 12
4.2. Step By Step Explanation 13
4.2.1. Image Capturing 13
4.2.2. Grayscale conversion 14
4.2.3. Automatic Code Detection 15
4.2.4. Pattern Matching 17
4.2.5. Internal Camera Calibration 21
4.2.6. Exterior Orientation 22
4.2.7. Generating 3D Coordinate Points 24
5. Implementation 25
5.1. Code 25
5.2. Graphical User Interface 26
6. Conclusion and Future Scope 32
7. Limitations 32
8. References 33

6
List of Figures and Illustrations
Sl .No Figure Description Page Number
Figure1 3-D Wire-Frame Models 8
Figure2 Hardware and Software
Requirements
11
Figure3 Flowchart of entire Algorithm 12
Figure4 Nikon D-80 Camera used for
capturing images
13
Figure5 Image Capturing Algorithm 13
Figure6 Captured Images 14
Figure7 Flowchart for Grayscale
Conversion
14
Figure8 Grayscale Images 15
Figure9 Flowchart for Dot Detection 16
Figure10 Dot Detected Images 17
Figure11 Failure of Delaunay Triangles 18
Figure12 Possible Position of Codes 19
Figure13 Image showing Rows on the
basis of distance
20
Figure14 Image showing columns of
the basis of distance
20
Figure15 Summary of Internal Camera
Calibration
22
Figure16 Summary of External Camera
Calibration
22
Figure17 Screenshot of the output 24
Figure18 3D Coordinate Points Plotted 25

7
1. ABSTRACT
The project report is entitled “Coded target detection and 3D Coordinate Points Generation”. As
its name goes, the objective of the project is to generate 3 dimensional coordinate points from a
set of 2D images. The entire project is based on the principles of Photogrammetry. The subject of
interest is embedded with patterns of retro-reflective dots. The reason that these dots were
attached is that they appear very bright in the image if it is captured using flash. Several pictures
of that subject were taken at different angles with the same focal length maintaining an overlap
of at least 70%. An auto-bar is used to define the coordinate system reference.
The algorithm which performs this transformation is broken down to several steps. First step is
converting the images to grayscale. Then detecting the retro-reflective dots by the “Polygon”
approach. After detection of the dots in several images, same dots in different images were
matched by Pattern Matching algorithm which is a significant step in the whole process. Then
internal camera calibration was performed which produced the six internal camera parameters
namely focal length, tangential distortion, radial distortion etc. Using this information, the
external camera orientation is found out. Finally, the 3D Coordinate points were generated by the
“Bundle Adjustment” procedure.
2. INTRODUCTION
2.1. Theory and Overview
Photogrammetry is the science of making measurements from photographs, especially for
recovering the exact positions of surface points.
The fundamental principle used by photogrammetry is triangulation. By taking photographs from
at least two different locations, lines of sight can be developed from each camera to points on the
object. These lines of sight are mathematically intersected to produce the 3-dimensional
coordinates of the points of interest.
The output of photogrammetry is typically a map, drawing, measurement, or a 3D model of some
real-world object or scene. Many of the maps we use today are created with photogrammetry and
photographs taken from aircraft. Moreover, it may be used to recover the motion pathways of
designated reference points located on any moving object, on its components and in the
immediately adjacent environment. Photogrammetry may employ high-speed imaging and
remote sensing in order to detect, measure and record complex 2-D and 3-D motion fields.
Its applications include satellite tracking of the relative positioning alterations in all Earth
environments (e.g. tectonic motions etc.), the research on the swimming of fish, of bird or insect
flight, other relative motion processes. The quantitative results of photogrammetry are then used
to guide and match the results of computational models of the natural systems, thus helping to
invalidate or confirm new theories, to design novel vehicles or new methods for predicting
or/and controlling the consequences of earthquakes, tsunamis, any other weather types, or used
to understand the flow of fluids next to solid structures and many other processes.
 Types of Photogrammetry

8
Photogrammetry can be classified a number of ways but one standard method is to split the field
based on camera location during photography. On this basis we have Aerial Photogrammetry,
and Close-Range Photogrammetry.
In Aerial Photogrammetry the camera is mounted in an aircraft and is usually pointed vertically
towards the ground. Multiple overlapping photos of the ground are taken as the aircraft flies
along a flight path. These photos are processed in a stereo-plotter (an instrument that lets an
operator see two photos at once in a stereo view). These photos are also used in automated
processing for Digital Elevation Model (DEM) creation.
In Close-range Photogrammetry the camera is close to the subject and is typically hand-held or
on a tripod. Usually this type of photogrammetry is non-topographic - that is, the output is not
topographic products like terrain models or topographic maps, but instead drawings, 3D models,
measurements and point clouds. Everyday cameras are used to model and measure buildings,
engineering structures, forensic and accident scenes, mines, earth-works, stock-piles,
archaeological artifacts, film sets, etc. This type of photogrammetry (CRP for short) is also
sometimes called Image-Based Modeling.
 Industrial Photogrammetry
Industrial Photogrammetry is a sub discipline of photogrammetry. To be very precise, it is
synonymous to Close-range Photogrammetry. But precision is of supreme importance in
Industrial Photogrammetry. It is an accurate, cost effective technique of collecting measurements
of a real world object or environment, directly from photographs. It adopts the theories and
methods of Computer technology, the digital image processing, image matching, pattern
recognition, etc. In recent years, close-range photogrammetry has developed quickly and has
become an effective and precise measuring tool for industrial metrology. 3D Modeling has
become extremely popular nowadays. There can be many ways to represent a 3D model which
include wireframe modeling, 3D point cloud generation and many more. Some 3D wireframe
models are shown below. These reveal how powerful these models can be for exploring the inner
finer structures of any object, howsoever complicated might it be.
Figure1:- 3-D Wire-frame models
Our project, though codenamed “Coded Target Detection and 3D Modeling” does not actually
produce 3D Wireframe model or generate 3D point cloud. Instead, it finds out the 3D coordinate
points of the retro-reflective dots attached to the subject of interest.

9
A retro-reflective dot appears very bright in the image if it is captured with flash. Actually, the
concerned object is embedded with patterns of retro-reflective dots. Some of these patterns are
shown below
2.2. Documents and Research Papers
For completion of our project, we had an extensive study of research papers and various online
as well as offline documents. Some of these helped us very much in this project and need special
mentioning.
 Research Papers
 Automatic Camera Calibration in Close Range Photogrammetry by Clive S. Fraser
(University of Melbourne) – This Paper discusses the impact of automation which has
greatly simplified the calibration tasks and the functional model adopted for self
calibration. Issues discussed include interior orientation stability, calibration reliability,
focal plane distortion, image point distribution, variation in lens distortion with image
scale, etc.
 Automatic Detection and Decoding of Photogrammetric Coded Targets by Udaya
Wijenayake, Sung-In Choi and Soon-Yong Park (School of Computer Science
Engineering, Kyungpook National University, South Korea)- this paper introduces an
automated coded target detection method which can be used to enhance the
efficiency of the Photogrammetry.
 An Implementation of Camera Calibration Algorithms by Meredith Drennan
(Department of Electrical and Computer Engineering, Clemson University) - This paper
seeks to provide an introduction to camera calibration procedures. It also discusses an
implementation of automating point correspondences in known planar objects. Finally,
results of a new implementation of Zhang‟s calibration procedure are compared to other
open source implementation results.
 Online Documents
 MATLAB® Creating Graphical User Interfaces (R2014a) by Math works. (The
MathWorks, Inc. 3 Apple Hill Drive Natick, MA)
 Offline Documents
 Building a Matlab GUI by Todd Wittman
 Graphics and GUIs with MATLAB®(3rd
Edition) by Patrick Marchand and O. Thomas
Holland
 A Guide to MATLAB for beginners and Experienced Users By Brian R. Hunt, Ronald L.
Lipsman and Jonathan M. Rosenberg
 Introduction on Photogrammetry by Koert Sijmons (ITC, Netherlands)

10
2.3. Organization of the report
The entire report has been divided into several logical sections.
 Initially, Abstract gives brief information about the project summarizing its main aspects.
 The Second section starts with Introduction which includes information about the basic
principles of Photogrammetry with special mention to Close Range Photogrammetry. The
various research papers and documents which were referred for the project are also listed.
 The Third section consists of Problem Statement and Detailed Problem Analysis. The
Proposed methodology is mentioned. The Hardware and Software requirements for our
software bundle to run are written.
 The Fourth section is divided into 7 sub-sections namely Image Capturing, Grayscale
conversion, Automatic Code Detection, Pattern Matching, Internal Camera Calibration,
Exterior Orientation and generating 3D Coordinate Points. The Image capturing instructs
the user on how to capture the images. As the name suggests Grayscale Conversion deals
with the conversion of images to grayscale. In Automatic Code Detection, the process of
how the retro-reflective dots are identified is described in detail. The Pattern Matching
deals with the automatic matching of the retro-reflective dots which is further used in the
generation of the 3D coordinate points. In Camera Calibration, the intrinsic parameters of
the camera such as focal length, radial distortion, angular distortion, scale factor and
principal point are computed. With the help of these parameters, the Exterior orientation
of the camera is calculated.
 The Fifth section covers the implementation details which provide the final design step of
the module. The various tables formed are also provided.
 The sixth and Seventh section comprises of the results and conclusions that are drawn
from the algorithm so framed. This is the final output of the process.
 Concluding all this will be the Summary and Limitations section which will round up the
entire discussion. It includes summary of achievements, main difficulties encountered
and how they were tackled, and future scope of the project.
 This will be followed by the reference/bibliography section that will tell about the books,
papers and sites that have been referred by us while developing the model

11
3. PROBLEM STATEMENT AND ANALYSIS
3.1. Problem Statement
The problem is to generate 3D coordinate points of any Industrial Object. By Industrial Object,
we mean man-made objects ranging from simple ones like box, glass, cones, cylinders etc. to
very complicated ones like the jet engine architecture of a Boeing all of which have regular
geometric shapes. The subject is embedded with retro-reflective dots. A set of two dimensional
photographs of the concerned subject are given.
3.2. Detailed Problem Analysis and Solution Strategy
A set of 2D images of any object embedded with retro-reflective in them were given to generate
3D model it so as to ultimately match it with its ideal model, i.e. its intended design, so as to
remove any glitches/defects in the real-time object.
Following steps are to be followed in order to achieve the objective-
1. Conversion of images to grayscale so as to reduce the processing size and increase the
contrast between the image and the targets.
2. Detection of dots using polygon approach in order to accurately determine their shape
in any orientation.
3. Camera calibration to compute internal camera parameters which are to be used later
for exterior orientation.
4. Pattern matching among coded targets of various images as per their orientation,
using concepts of slope and distance.
5. Computation of exterior or spatial orientation of the camera.
6. Generation of the 3D model using the above two steps.
Accuracy to the sub-micron level is to be achieved
3.3. Hardware and Software requirements
The Software Bundle that we developed requires the following features to run smoothly on a
computer.
Minimum Requirement Recommended
Hardware
 Intel Pentium Processor
 1GB of RAM
 1GB of free Hard Disk Space
 Intel core i5 or above
 4GB of RAM or more
 2GB of free Hard Disk Space or more
Software
 Windows XP ( We strongly
recommend not to use Windows XP on
your Personal Computer as Microsoft
stopped providing support for this
Operating System.)
 Windows 7 or 8

12
 MATLAB 2012a  MATLAB 2013a
Fig2:- Hardware and Software Requirements
4. ALGORITHM DESIGN
4.1. The Flow Chart
The entire algorithm can be broken down into several steps as written below.
 Capturing the images of the subject. The subject is embedded with patterns of retro-
reflective dots. The images thus captured are in the RGB .jpeg format.
 Converting the RGB images to Grayscale images.
 Detecting the retro-reflective dots in the image.
 Matching the dot patterns across the different images.
 Calibrating the camera so as to find the internal camera parameters.
 Finding the External camera orientation.
 Using Photogrammetric Equations, finding out the 3D coordinates of the concerned
subject.
The workflow can be represented by the following flow chart.
Fig3:- Flowchart of the entire Algorithm
Start image capturing
converting to
grayscale
detection of retro
reflective dots by
polygon approach
automated pattern
matching
camera calibration
to find out the
internal parameters
using
photogrammetric
equations to find
the 3D coordinates
to plot 3D
coordinate points
end

13
4.2. Step by Step explanation
4.2.1. Capturing the images
The first step is to capture the images of the subject. Nikon D-80 Digital Single Lens Reflecting
camera was used to capture the images. The subject of interest was a cardboard box. It was
embedded with retro-reflective dots. A retro-reflective dot appears very bright in the image when
it is captured using flash. Various images were captured at differing angles.
However while taking the images; the user must keep three points in mind.
 The subject of interest should not be captured in much tilted positions as it will lead to erroneous
results.
 The images must be taken using the flash, otherwise the retro-reflective dots won‟t appear too
bright to be detected.
 All the images must be captured at fixed focal length.
Fig4:-Nikon D-80 DSLR which is used to take the images
The process of taking the images can be shown in the following flowchart.
Fig5:- Image Capturing Algorithm
The captured images looked like this.
The concerned subject is embedded with retro reflective
dots.
Images were captured by Nikon D-80 DSLR at various
angles using flash keeping the same focal length.
The retro reflective dots appeared very bright as
compared to other parts of the image.

14
Fig6. Captured images
4.2.2. Grayscale Conversion
The captured images were initially in RGB. Before any further processing, it is necessary to convert these
into grayscale. The standard MATLAB function RGB2GRAY was used to convert the images.
Fig7:-Flowchart for Grayscale conversion
After conversion to grayscale, the images looked like this.
RGB image
Using
standard
MATLAB
function
RGB2GRAY
Grayscale
image

15
Fig8. Grayscale images
4.2.3. Dot Detection
After the grayscale conversion is over, the process of detecting the dots started. For detecting the
retro-reflective dots, the Polygon approach is used.
Why this approach?
The images were captured at varying angles and the retro reflective dots were appearing in
various shapes (mostly ellipse) and sizes. As a result, these dots were not exactly circular and the
standard command IMFINDCIRCLES could not detect the centers accurately. Moreover, the
edges of the dots were not sharp enough.
The polygon approach can be broken down to several steps which are,
 First, we used IMFINDCIRCLES to estimate the approximate center and radius of the
dots.
 Then, a concentric circle with a greater radius is plotted such that the entire shape is
inside the outer circle.
 The entire outer circle was searched for the white pixels which were stored in a matrix.
 The standard CONVHULL function of MATLAB is used to recognize the exact
polygonal shape of the dot.

16
 The polygon was plotted along with the images.
 After that, the centroids of the polygons were found out.
We can represent the entire algorithm in a flowchart.
Fig9:-Flowchart illustrating dot detection
The dots that were detected in the images looked like this.
• Used
IMFINDCIRCLES to
detect the
approximate centers
& radius of the dots.
Finding circles
•Plotted a concentric
circle with a greater
radius using
VISCIRCLES
•Searched for all the
white pixels within
the outer circle
Reducing
inaccuracy
•Found the
Polygon using
CONVHULL
function
Finding
Polygon

17
Fig10. Dot detected images
4.2.4. Pattern Matching
Once the dots were detected it was now required to identify the centroid of the dots of the coded
targets. For this the following formula was been used.
The centroid of a non-self-intersecting closed polygon defined by n vertices (x0, y0 ), (x1,y1), ...,
(xn−1,yn−1) is the point (Cx, Cy), where
After the centroids were detected different algorithms and methods were being worked upon to
match the set of centroids in each coded target with the corresponding one in another image.
 Previous approach
1) Delaunay Triangulation

18
It is used to create a 2-D or 3-D triangulation from a set of points. In a 2-D Delaunay
triangulation, the circumcircle associated with each triangle does not contain any points
in its interior.
Though it is one of the finest methods but it didn‟t work in our images i.e. the Delaunay
Pattern was not same in two of the images which are shown below: (the diagonal of the
square is different in both the images)
Fig11:- Failure of Delaunay Triangulation
 Our Approach
Detection of 8 points in order :-
1) First the center of coded target was found using distance
2) After finding the center the main diagonal was isolated from all the present diagonal in the
coded target
3) After this the two symmetric points were found out
4 ) After detecting the above 5 mentioned points in the coded target the main 3 points were
detected .
5 ) After detecting the rest three points the were stored in an array in a sorted manner on the
basis of x - coordinate
6) After this all the 8 points of coded target were stored in an array in the following order
1 -> first point of diagonal having minimum distance from center ( YELLOW )
2 -> point which was right to the firt point ( PURPLE )
3 -> point which was left to the firt point ( BLACK )
4 -> center of coded target ( RED )
5 -> second point of diagonal ( BLUE )
6 , 7 , 8 - > points sorted on x coordinate basis
6 ( CYAN ) 7 ( WHITE ) 8 ( GREEN )

19
Detection of autobar
1) First the 6 points of autobar were found out
2) After this the found points which were lying on line were isolated
3) After this the two symmetric points were stored on the basis of rotation of the autobar
COLORING OF AUTOBAR :
1 - > WHITE
2 -> RED
3 - > CYAN
4 - > YEllOW
5 -> BLACK
6 ->GREEN
Segmentation of Circular Dots in Coded Targets
This method is based on the simple concepts of ratios and distances.
Fig12:- Possible positions of codes
The entire methodology can be broken down in the following steps:-
 First on the basis of slopes and ratios, new coordinates axis – x-axis, y-axis and the
diagonal axis are plotted.
 Then on the basis of ratios of the distances of the various points rows are marked as
following.
0
5
10
15
20
25
30
35
0 10 20 30 40
Series1

20
Fig13:- Image showing the rows on the basis of distances
 Using distance formula, perpendicular distances of the three points are calculated from
these row lines.
 The row line to which the points are nearest is the corresponding row of the points.
 For the allotted rows, ratios of columns were used and the absolute differences of the
distance of these unknown points were obtained. The one which was the smallest was
taken.
Fig14:- Image showing the columns and the differences of the distances
Matching of Coded Targets
• Each image has n number of coded targets. All the center coordinates of the one target
are stored in a file.

21
• Each coded target has 8 center coordinates. 4 of them form a square and 5th one is on the
diagonal.
• The remaining three can lie on either side of the diagonal. There can be at most two
targets on either side of the diagonal.
• If a side of a diagonal contains one point, then the other point on that side of the diagonal
is given value -1.
• The row and column of the 4 positions, two on either side of the diagonal, are calculated
for each coded target and stored in a file.
Storing the set number of Coded Targets
• First those sets are chosen in which 8 points are completely matched. Then in next round
of matching, the remaining sets are matched to the most suitable one.
• If there are less than 5 points matched for a particular set, it will show the message for
image to be discarded or user to choose the rows and column for that particular set.
Storing the Center Coordinates
• Center Coordinates are extracted from their respective center coordinates file and stored
in a file.
4.2.5. Internal Camera Calibration
This entire process finds out the five internal camera parameters namely focal length, scale
factor, radial distortion, angular distortion and principal point.
A calibration procedure for accurately determining the pose and internal parameters of several
cameras is described. Multiple simultaneously-captured sets of images of a calibration object in
different poses are used by the calibration procedure. Coded target patterns, which serve as
control points, are distributed over the surface of the calibration object. The observed positions
of these targets within the images can be automatically determined by means of code band
patterns. The positions of the targets across the multiple images are then used to infer the camera
parameters.
 The code for camera calibration algorithm was originally created by Janne Heikkila, University of
Oulu, Finland.
 This program takes camera specifications, a matrix consisting of 3D coordinates, image
coordinates and the normal vector of control points and outputs the intrinsic camera parameters
which are focal length, radial distortion, principle coordinates, angular distortion and scale factor.
The process can be shown in a flowchart.

22
Fig15:-Summary of Internal Camera Calibration
The input matrix has nx8 data elements which serve as the image co-ordinates of the control points.
Initially the code was made to run on two images which gave the focal length of about 19.37. As we
increase the number of images, we can get more accurate results with standard error in pixels of about
0.000278. The rough elapsed time is about .1 seconds.
4.2.6. Exterior Camera Orientation
As the name suggests, this process finds out the camera orientation. Camera orientation is
nothing but it‟s 3D spatial coordinates along with three spatial angles totaling six parameters.
The entire algorithm can be shown in the following smart-art.
Fig16:- Summary of External Camera Calibration
The exterior parameters of the camera are the three dimensional co-ordinates of the camera and
also the angular orientation of the camera while the image is being captured. The main objective
of this section is to establish a relationship between the digital image(pixel) co-ordinate system
and the real world (latitude and longitude) co-ordinate system. For achieving this, the
collinearity equations were used.
The collinearity equations are a set of two equations used in photogrammetry and remote
sensing used to determine the object coordinates( three dimensions, namely the X, Y, Z of the
1. Camera Properties
like Focal length, CCD
Chip size, Pixels in
Horizontal & vertical
Direction
2. Image Matrix
containing spatial
coordinates, image
coordinates and
normal vector of the
control points.
Inputs
The internal camera
parameters
1. Focal length
2. Angular
distortion
3. Radial distortion
4. Scale factor
5. Principal point
Outputs
The internal camera
parameters
1. Focal length
2. Angular distortion
3. Radial distortion
4. Scale factor
5. Principal point
input
3D spatial
coordinates of the
subject.
output

23
object) using the coordinates in the sensor plane.( two dimensions). The collinearity equations
are given as follows:
In the above Equation the parameters involved are :
 Rij (i=1,2,3 ; j=1,2,3) are the elements of the rotation matrix.
 ( X, Y , Z ) are the 3-D global coordinates of the object
 x, y are the image co-ordinates of the object.
 c is a constant which is determined initially.
The above equations describes the transformation from the object space (X,Y,Z) to the image
coordinates (x , y) . For implementing the process in MATLAB, codes were obtained from the
website of ITC (University of Twente) , Netherlands. The codes obtained were based on the
two basic principles:
1. Newton's Method: It is an iterative method which is used for determining the root of any
given real polynomial or any real transcendental function. It is mainly used in numerical
analysis, and also finds extensive applications in other fields.
2. Least Square Method: The method of Least squares is a standard approach to
approximate solutions of over determined systems, that is, systems in which there are
more equations than the unknown parameters. It is a method which is extensively used
in regression analysis, and is one of the fundamental principles in determining the
exterior parameters of the camera ,which then help in determining the 3-D co-ordinates of
any desired point of the object.
How the Code works:
The code used is used to obtain the spatial orientation of the camera, while the image is
being captured. Initially, the input parameters of the code mainly include the focal length
of the camera, which is obtained from the internal camera calibration process. Also, a set
of random values of the exterior parameters ( Approximate values obtained by trial and
error method) are used as input arguments. The output consists of the adjusted exterior
parameters of the camera ( which are the correct parameters) and also a graph showing
the correction of orientation angles vs. number of iterations to reach the final output.
The code mainly uses the Newton's Method along with the Least Square Method to
iterate the initially obtained values to calculate the final adjusted exterior parameters,
which are the correct parameters.
A screenshot showing the final output of the exterior parameters is shown as follows:

24
Fig17:- The screenshot of the output of the exterior parameters section.
Thus, from the above graph we can clearly see that the exterior parameters section help us in
determining the correct spatial orientation of the camera which is then further used in
determining the 3-D co-ordinates of the coded targets.
4.2.7. Generating 3D Coordinate Points
• Bundle Adjustment
The Bundle Adjustment is the program that processes the photographic measurements to
produce the final XYZ coordinates of all the measured points. In order to do this, it must
triangulate the target points, resect the pictures and Self-calibrate the camera.
The real power of the bundle adjustment is that it is able to do all three of these things
simultaneously. If you review the descriptions of Triangulation and Resection, it appears there is
a problem. In order to triangulate the measured points, we must know the orientation of the
pictures.
However, in order to orient the pictures, we must know the coordinates of the measured points.
How do we get started here- The answer is the bundle adjustment has the capability to figure
them both out simultaneously and to self-calibrate the camera as well! This is where the name
bundle adjustment comes from because it bundles all these things together and solves them all at
the same time.
• How the code works:
The code used is to obtain the 3D metric coordinates of autobar and coded target points. This 3D
model generation links with all the pervious stages of the project. It takes the outputs of pattern
matching, camera calibration, exterior orientation as inputs. It works on the principle of
Newton‟s method (to solve collinearity equations) and least square adjustment method.
The camera positions of two images(x,y,z,w,p,k) which are obtained from exterior orientation,
focal length (obtained from camera calibration), autobar and coded target points in image
coordinate system (obtained from pattern matching) are given as inputs. The code is developed

25
in such a way that the output can be given for two images. In order to increase the accuracy,
combinations of n images taken twice are fed into the code and the average is taken.
.
Fig18: 3D coordinate points plotted
5. IMPLEMENTATION
5.1. Code Summary
 Convert to Grayscale:
1. Input: The original RGB image
2. Output: The image in the grayscale form
3. Description: The above function converts the original image into the grayscale
form.
 Dot Detection;
1. Input: The image in the grayscale form.
2. Output: The grayscale image with all the retro-reflective dots being detected and
their centers marked.
3. Description: The 'Dot Detection' function detects all the retro-reflective dots in the
grayscale image which is being given as input within a particular radius range.
The Autobar is also detected and separately marked.
 Camera Calibration:
1. Input: The image with the Autobar and the dots detected in them.
2. Output: The internal camera parameters which includes the focal length, scale
factor, angular distortion etc.
3. Description: The above function calculates the internal camera parameters using
the centers of the Autobar which were initially detected.
 Pattern Matching:
1. Input: The image with the centers of the coded targets and the Autobar being
detected.

26
2. Output: A text file named 'finalcord.txt' with the retro-reflective dots in all the
images being matched.
3. Description: The Function matches the retro-reflective dots in all the images
which have been initially given as input.
 Exterior Orientation:
1. Input: The internal camera parameters along the Autobar co-ordinates.
2. Output: The external parameters of the camera which includes the spatial and
angular orientation of the camera.
3. Description: The function calculates the external camera parameters of the
camera.
 3-D co-ordinate Points:
1. Input: The focal length, the external parameters along with the matched coded
targets.
2. Output: The figure showing the 3-D co-ordinate points of all the retro-reflective
dots.
3. Description: The function calculates and plots the 3-D co-ordinate points of all the
retro-reflective dots.
5.2. GUI
The software generates the 3-D coordinates of the object from a given set of its 2-D images.
Follow the steps mentioned below-
1. Click some images of the subject.
 Note that the subject must be embedded with the Retro-Reflective targets. The
images must be captured with the flash on.
 Images should be clicked from a distance of about 1m.
 Beware not to click the images in much tilted positions.
2. Open the GUI and click on „Run‟ button ( ) or press (Ctrl+T).
 A popup box appears with options to create a new project or open an existing
project or get help or access the web.

27
 If „Create a New Project‟ option is selected, enter the name of the project you
want to create. A new folder of the given name is created in the specified
directory.
 All the files henceforth generated by the software will be saved in the same
folder.
3. Load the images
 Enter the number of images, to be processed, in the edit box.
 Click on „Import Image‟ button. A „Select Image‟ Window appears. Select the
images from any directory.

28
Convert the images to Grayscale using the Convert to Grayscale Button.
Arrow 1 shows the list of original images.
Arrow 2 shows the list of grayscale images.
Images appended in these list boxes can be viewed individually on the screen by selecting them.
Also, the „.txt‟ files which store the center coordinates of the coded targets of the images can be
viewed from here.
4. Detect the Retro-Reflective Dots using „Dot Detection„ Button.
 You need to enter the minimum and maximum possible radius of the Retro-
Reflective Dots as appearing in your images. The maximum radius should be less
than 3 times the minimum radius.
1
2

29
Centers of the coded targets are stored in the matrix which can be viewed as soon as dot
detection is completed for all the images or by selecting the matrix of any particular image from
the list box as indicated by the arrow in the above image
5. Click on „Camera Calibration‟ button.
 The first matrix that appears on the screen gives the image matrix, indicated by
Arrow 1.

30
 The pop up box that appears next gives internal parameters of the camera,
indicated by Arrow 2. These along with exterior camera parameters are used to
generate 3D model.
6. Click on “Pattern Matching‟ button.
 It matches coded targets in different locations of the images so loaded.
2
1

31
7. Click on „Exterior Orientation‟ button.
 These graphs show correction of orientation angles with the number of iterations.
Least squares method is followed.
 The popup box shows external camera parameters.
9. Click on „3D Coordinates’ button.
 All the 3D coordinates will be shown in a pop up window.
 User can rotate as he needs with the rotation tool.

32
6.CONCLUSIONS AND FUTURE SCOPE
FUTURE SCOPE-
• The final co-ordinates of the coded targets can be found out by using a 3-D conformal
transformation.
• For obtaining the 3-D model, the co-ordinates obtained in the output are plotted and the
points are densified to generate the required 3-D model.
7. LIMITATIONS
There are some minor limitations of the software developed during the project.
 The subject of interest must not be captured in much tilted positions (at the minimum
angle of 60 degrees with the plane in which the object is kept). The dot detection
algorithm will otherwise detect more than one circle for the same coded target. In some
cases, it might not detect any circles for some coded targets. Hence, rejection of such
images should be done manually.
 The image size should be no less than 3872 X 2592 pixels in dimensions. Otherwise the
dots would not be sharp enough to be recognized by the software. However it depends on
the size of the retro-reflective dots attached in the subject and the way these appear in the
images.
 The user should input minimum and maximum possible radius of the retro-reflective dots
as appearing in the image in pixels manually. User also needs to enter segmentation
distance in pixels which is the average distance between two coded patterns.
1
• The basic principles of Photogrammetry
2
• Camera Calibration
3
• Exterior Orientation of Camera
4
• 3D Coordinate Generation
5
• How to build a MATLAB GUI

33
8. REFERENCES
Online
 http://en.wikipedia.org/wiki/Photogrammetry
 http://www.photogrammetry.com/
 http://www.geodetic.com/v-stars/what-is-photogrammetry.aspx
 http://en.wikipedia.org/wiki/Camera_resectioning
 http://www.vision.caltech.edu/bouguetj/calib_doc/htmls/parameters.html
 http://www.umiacs.umd.edu/~ramani/cmsc828d/lecture9.pdf
 http://en.wikipedia.org/wiki/Direct_linear_transformation
 http://www.vision.caltech.edu/bouguetj/calib_doc/papers/heikkila97.pdf
 http://cronos.rutgers.edu/~meer/TEACHTOO/PAPERS/zhang.pdf
 Introduction on Photogrammetry-Koert Sijmons.
 www.wikipedia.org/wiki/Newton‟s_method
 www.wikipedia.org/wiki/Least_squares
 http://www.geodetic.com/v-stars/what-is-photogrammetry.aspx
 http://en.wikipedia.org/wiki/Photogrammetry
 http://www.photogrammetry.com/
 www.opticalengineering.spiedigitallibrary.org
 www.crcsi.com.au
 www.geodesic.com/v-stars/what-is-photogrammetry.aspx

project report final

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Viewers also liked

Viewers also liked (20)

Similar to project report final

Similar to project report final (20)

project report final