The RoboCV Workshop

and
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa
| Ajusal Sugathan & Utkarsh Sinha

presents

The word robot originally was supposed to
mean a slave
It is a machine which performs a variety
of tasks, either using manual external
control or intelligent automation
A manually controlled car or a ASIMOV
trying to kick a football are all robots
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha

ž Robotics is a multi disciplinary field of
engineering encompassing the vistas of
› Mechanical design
› Electronic control
› Artificial Intelligence
ž It finds it’s uses in all aspects of our life
› automated vacuum cleaner
› Exploring the ‘Red’ planet
› Setting up a human colony there :D

ROBOTS
CONTROL
AUTONOMOUS
MANUAL
APPLICATIONS
INDUSTRIAL
MEDICAL
INTERFACE
HARDWARE
SOFTWARE
INTERLINKED

Ø Locomotion System
Ø Actuators
Ø Power Supply System
Ø Transmission System
Ø Switches
Ø Sensory Devices For Feedback
Ø Sensor Data Processing Unit

Ø A mobile robot must have a system to
make it move. Ob.
Ø This system gives our machine the ability
to move forward, backward and take
turns
Ø It may also provide for climbing up and
down
Ø Or even flying or floating J

Ø Each type of locomotion requires
different number of degrees of freedom
Ø More degrees of freedom means more the
number of actuators you will have to use
Ø Although one actuator can be used to
control more than one degree of freedom

Ø Wheeled
Ø Legged
Ø Climbing
Ø Flying
Ø Floating
Ø Snake-Like

Ø The kind of locomotion most frequently
used in robotics at the undergrad level
Ø This involves conversion of electrical
energy into mechanical energy (mostly
using motors)
Ø The issue is to control these motors to
give the required speed and torque

Ø We have a simple equation for the
constant power delivered to the motor:
› P = ζ X ω
Ø Note that the torque and angular velocity
are inversely proportionally to each other
Ø So to increase the speed we have to
reduce the torque

Ø The dc motors available have very high
speed of rotation which is generally not
needed
Ø At high speeds, they lack torque
Ø For reduction in speed and increase in
“pulling capacity” we use pulley or gear
systems

Ø Differential Drive
Ø Dual Differential Drive
Ø Car-type Drive
Ø Skid-steer Drive
Ø Synchronous Drive

Ø Simplest, easiest to implement and most
widely used.
Ø It has a free moving wheel in the front
accompanied with a left and right wheel. The
two wheels are separately powered
Ø When the wheels move in the same direction
the machine moves in that direction.
Ø Turning is achieved by making the wheels
oppose each other’s motion, thus generating
a couple

Ø In-place (zero turning radius) rotation is
done by turning the drive wheels at the same
rate in the opposite direction
Ø Arbitrary motion paths can be implemented
by dynamically modifying the angular
velocity and/or direction of the drive wheels
Ø Total of two motors are required, both of
them are responsible for translation and
rotational motion

Ø Simplicity and ease of use makes it the most
preferred system by beginners
Ø Independent drives makes it difficult for
straight line motion. The differences in
motors and frictional profile of the two
wheels cause them to move with slight
turning effect
Ø The above drawback must be countered with
appropriate feedback system. Suitable for
human controlled remote robots

Ø Uses synchronous rotation of its wheels to
achieve motion & turns
Ø It is made up of a system of 2 motors.
One which drive the wheels and the other
turns the wheels in a synchronous fashion
Ø The two can be directly mechanically
coupled as they always move in the same
direction with same speed

The direction of motion is given by
black arrow. The alignment of the
machine is shown by red arrow

Ø The use of separate motors for translation
and wheel turning guarantees straight
line motion without the need for dynamic
feedback control
Ø This system is somewhat complex in
designing but further use is much simpler

Ø Actuators, also known as drives, are
mechanisms for getting robots to move.
Ø Most actuators are powered by
pneumatics
(air pressure), hydraulics (fluid pressure),
or motors (electric current).
Ø They are devices which transform an
input signal (mainly an electrical signal))
into motion

Ø Widely used because of their
small size and high energy output.
Ø Operating voltage: usually 6,12,24V.
Ø Speed: 1-20,000 rpm..
Ø Power: P = ζ X ω

ØThe stator is the
stationary outside part of a
motor.
Ø The rotor is the inner
part which rotates.
Ø Red represents a
magnet or winding with a
north polarization.
Ø Green represents a
magnet or winding with a
south polarization.
Ø Opposite, red and
green, polarities attract.
Ø Commutator contacts
are brown and the brushes
are dark grey.

Ø Stator is composed of two or more
permanent magnet pole pieces.
Ø Rotor composed of windings which are
connected to a mechanical commutator.
Ø The opposite polarities of the energized
winding and the stator magnet attract and the
rotor will rotate until it is aligned with the
stator.
Ø Just as the rotor reaches alignment, the
brushes move across the commutator contacts
and energize the next winding.
Ø A yellow spark shows when the brushes
switch to the next winding.

ØIt is an electric
motor that can divide a
full rotation into a large
number of steps.
Ø The motor's position
can be controlled
precisely, without any
feedback mechanism.
Ø There are three
types:
Ø Permanent
Magnet
Ø Variable
Resistance
Ø Hybrid type

Ø Stepper motors work in a similar way to
dc motors, but where dc motors have 1
electromagnetic coil to produce movement,
stepper motors contain many.
Ø Stepper motors are controlled by
turning each coil on and off in a sequence.
Ø Every time a new coil is energized, the
motor rotates a few degrees, called the
step angle.

Full Step
Ø Stepper motors have 200
rotor teeth, or 200 full steps
per revolution of the motor
shaft.
Ø Dividing the 200 steps
into the 360º's rotation
equals a 1.8º full step angle.
Ø Achieved by energizing
both windings while
reversing the current
alternately.

ØServos operate on the principle of negative
feedback, where the control input is compared to
the actual position of the mechanical system as
measured.
ØAny difference between the actual and wanted
values (an "error signal") is amplified and used to
drive the system in the direction necessary to
reduce or eliminate the error
ØTheir precision movement makes them ideal for
powering legs, controlling rack and pinion steering,
to move a sensor around etc.

Ø Suitable power source is needed to run
the robots
Ø Mobile robots are most suitably powered
by batteries
Ø The weight and energy capacity of the
batteries may become the determinative
factor of its performance

Ø For a manually controlled robot, you can
use batteries or voltage eliminators
(convert the normal 220V supply to the
required DC voltage 12V , 24V etc.)

Ø Gear
Ø Belt Pulley
Ø Chain Sprocket
Ø Rack and Pinion
Ø Pick Place Mechanisms

Ø Gears are the most common means of
transmitting power in mechanical
engineering
Ø Gears form vital elements of mechanisms in
many machines such as vehicles, metal
tooling machine tools, rolling mills, hoisting
etc.
Ø In robotics its vital to control actuator speeds
and in exercising different degrees of
freedom

Ø To achieve torque magnification and
speed reduction
Ø They are analogous to transformers in
electrical systems
Ø It follows the basic equation:
Ø ω1 x r1 = ω2 x r2
Ø Gears are very useful in transferring
motion between different dimension

Ø An arrangement of gears to convert
rotational torque to linear motion
Ø Same mechanism used to steer wheels
using a steering
Ø In robotics used extensively in clamping
systems

Ø It allows for mechanical power, torque,
and speed to be transmitted across axes
Ø If the pulleys are of differing diameters, it
gives a mechanical advantage
Ø In robotics it can be used in lifting loads
or speed reduction
Ø Also it can be used in a differential drive
to interconnect wheels

Ø Sprocket is a profiled wheel with teeth that
meshes with a chain
Ø It is similar to the system found in bicycles
Ø It can transfer rotary motion between shafts
in cases where gears are unsuitable
Ø Can be used over a larger distance
Ø Compared to pulleys has lesser slippage due
to firm meshing between the chain and
sprocket

Ø For picking and placing many mechanisms
can be used:
vHook and pick
vClamp and pick
vSlide a sheet below and pick
vMany other ways
vLots of Scope for innovation

ž Image Processing is a tool for
analyzing image data in all areas of
natural science
ž It is concerned with extracting data
from real-world images
ž Differences from computer graphics is
that computer graphics makes
extensive use of primitives like lines,
triangles & points. However no such
primitives exist in a real world images.

ž Increasing need to replicate human
sensory organs
ž Eye (Vision) : The most useful and
complex sensory organ

ž Automated visual inspection system
Checking of objects for defects visually
ž Remote Sensing
ž Satellite Image Processing
ž Classification (OCR), identification
(Handwriting, finger prints) etc.
ž Detection and Recognition systems
(Facial recognition..etc)
ž Biomedical applications

ž Camera, Scanner or any other image
acquisition device
ž PC or Workstation or Digital Signal
Processor for processing
ž Software to run on the hardware platform
(Matlab, Open CV etc.)
ž Image representation to process the
image (usually matrix) and provide spatial
relationship
ž A particular color space is used to
represent the image(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha

Image Acquisition Device
(Eg. CCD or CMOS Camera)
Image Processor
(Eg. PC or DSP)
Image Analysis Tool
(Eg. Matlab or Open CV)
Machine Control Of Hardware
through serial or parallel interfacing

ž Using a camera
ž Analog cameras
ž Digital cameras
› CCD and CMOS cameras
ž Captures data from a single
light receptor at a time
ž CCD – Charge Coupled Devices
ž CMOS – Complementary MOSFET Sensor
based

ž Digital Cameras
› CCD Cameras
– High quality, low noise images
– Genarates analog signal converted using ADC
– Consumes high power
› CMOS Cameras
– Lesser sensitivity
– Poor image quality
– Lesser power
ž Analogue cameras require grabbing card
or TV tuner card to interface with a PC

Colored pixels on CCD
Chip

ž Matlab
ž Open CV

ž Two types: Vector and Raster
ž Vector images store curve information
ž Example: India’s flag
ž Three rectangles, one circle and
the spokes
ž We will not deal with vector images at
all

ž Raster images are different
ž They are made up of several dots

ž If you think about it, your laptop’s
display is a raster display
ž Also, vector images are high level
abstractions
ž Vector representations are more
complex and used for specific purposes

ž Raster
› Matrix
ž Vector
› Quadtrees
› Chains
› Pyramid
Of the four, matrix is the most general.
The other three are used for special
purposes. All these representations must
provide for spatial relationships

ž Computers cannot handle continuous
images but only arrays of digital
numbers
ž So images are represented as 2-D
arrays of points (2-D matrix)(Raster
Represenatation)
ž A point on this 2-D grid (corresponding
to the image matrix element) is called
PIXEL (picture element)
ž It represents the average irradiance
over the area of the pixel

ž Each pixel requires some memory
ž Color depth : Amount of memory each
pixel requires
ž Examples
› 1-bit
› 8-bit
› 32-bit
› 64-bit

ž Pixels are tiny little dots of color you see on
your screen, and the smallest possible size
any image can get
ž When an image is stored, the image file
contains information on every single pixel in
that image i.e
› Pixel Location
› Intensity
ž The number of pixels used to represent the
image digitally is called Resolution
ž More the number of pixels used, higher the
resolution
ž Higher resolution requires more
processing power

ž MATLAB stands for MATrix LABoratory,
a software developed by Mathworks
Inc (www.mathworks.com). MATLAB
provides extensive library support for
various domains of scientific and
engineering computations and
simulations

ž When you click the MATLAB icon (from
your desktop or Start>All Programs),
you typically see three windows:
Command Window, Workspace and
Command History. Snapshots of these
windows are shown below

ž This window shows the variables
defined by you in current session on
MATLAB

ž Command History stores the list of
recently used commands for quick
reference

ž This is where you run your code

ž In MATLAB, variables are stored as
matrices (singular: matrix), which
could be either an integer, real
numbers or even complex numbers
ž These matrices bear some
resemblance to array data structures
(used in computer programming)

ž Let us start with writing simple
instructions on MATLAB command
window
ž To define an integer,
ž Type a=4 and hit enter
ž >>a=4
ž To avoid seeing the variable, add a
semicolon after the instruction
ž >>a=4;

ž Similarly to define a 2x2 matrix, the
instruction in MATLAB is written as
ž >> b=[ 1 2; 3 4];
ž If you are familiar with operations on
matrix, you can find the determinant
or the inverse of the matrix.
ž >> determin= det(b)
ž >> d=inv(b)

ž Images as we have already seen are
stored as matrices
ž So now we try to see this for real on
MATLAB
ž We shall also look into the basic
commands provided by MATLAB’s
Image Processing Toolbox

ž Once you have started MATLAB, type the
following in the Command Window
ž >> im=imread(‘sample.jpg');
ž This command stores the file image file
‘sample.jpg’ in a variable called ‘im’
ž It takes this file from the Current-
Directory specified
ž Else, entire path of file should be
mentioned

ž You can display the image in another
window by using imshow command
ž >>figure,imshow(im);
ž This pops up another window (called as
figure window), and displays the image
‘im’

ž The ‘imview’ command can also be
used in order toview the image
ž imview(im);
ž Difference is that in this case you can
see specific pixel values just by moving
the cursor over the image

ž To know the breadth and height of the
image, use the size function,
ž >>s=size(im);
ž The size function basically gives the
size of any array in MATLAB
ž Here we get the size of the IMAGE
ARRAY

ž Now that we have our image stored in
a variable we can observe and
understand the following:
ž How pixels are stored?
ž What does the values given by each
pixel indicate?
ž What is Image Resolution?

ž Have a look at the values stored
ž Say the first block of 10 x 10
ž >>im(1:10,1:10);
ž Or Say view the pixel range 50:150 on
both axis
ž >>
figure,imshow(im(50:150,50:150));

ž 1-bit = BLACK or WHITE
ž 8-bit = 28 different shades
ž 24-bit = 224 different shades
ž 64-bit images – High end displays
ž Used in HDRI, storing extra
information per pixel, etc

ž This is another name for 1-bit images
ž Each pixel is either White or Black
ž Technically, this is a black & white
image

ž Another name for 8-bit images
ž Each pixel can be one of 256 different
shades of gray
ž These images are popularly called
Black & White. Though, this is
technically wrong.

ž Again, each pixel gets 8 bits
ž But each of the 256 values maps to a
color in a predefined “palette”
ž If required, you can have different
bit depths

ž We won’t be dealing with indexed
images

ž 8-bits is too less for all the different
shades of colors we see
ž So 24-bits is generally used for color
images
ž Thus each pixel can have one of 224
unique colors

ž Now, a new problem arises:
ž How do you manage so many different
shades?
ž Programmers would go nuts
ž Then came along the idea of color
spaces

ž A color space can be thought of as a
way to manage millions of colors
ž Eliminates memorization, and
increases predictability
ž Common color spaces:
› RGB
› HSV
› YCrCb or YUV
› YIQ

ž You’ve probably used this already

ž Each pixel stores 3 bytes of data
ž The 24-bits are divided into three 8-bit
values
ž The three are: Red, Green and Blue i.e
the primary colours
ž Mixing of primary colours in right
proportions gives any particular colour
ž Each pixel has these 3 values

ž 1 byte = 8 bits can store a value
between 0-255
ž We get pixel data in the form RGB
values with each varying from 0-255
ž That is how displays work
ž So there are 3 grayscale channels

ž Advantages:
› Intuitive
› Very widely used
ž Disadvantages:
› Image processing is relatively tough

ž HSV makes image processing easier
ž Again, 24 bits = three 8-bit values or 3
channels
ž The 3 channels are:
› Hue
› Saturation (Shade of Colour)
› Value (Intensity)

ž The Hue is the tint of color used
› It represents the colour of the pixel (Eg. Red
Green Yellow etc)
ž The Saturation is the “amount” of
that tint
› It represents the intensity of the colour (Eg.
Dark red and light red)
ž The Value is the “intensity” of that
pixel
› It represents the intensity of brightness of the
colour

ž RGB image converted to HSV
RGB
HUE
SATURATION
VALUE

ž Advantages:
› The color at a pixel depends on a single
value
› Illumination independent
ž Disadvantages:
› Something

ž Intuitively RGB might seem to be the simpler
and better colour space to deal with
ž Though HSV has its own advantages
especially in colour thresholding
ž As the colour at each pixel depends on a
single hue value it is very useful in
separating out blobs of specific colours even
when there are huge light variations
ž Thus it is very useful in processing real
images taken from camera as there is a large
amount of intensity variation in this case
ž Hence, ideal for robotics applications

ž Widely used in digital video
ž Has three 8-bit channels:
› Y Component:
– Gives luminance or intensity
› Cr Component:
– It is the RED component minus a reference value
› Cb Component:
– It is the BLUE component minus a reference
value
ž Hence Cr and Cb components represent
the colour called “Color Difference
Components”

ž Advantages:
› Used in video processing
› Gives you a 2-D colour space hence helps
in closer distinguishing of colours
ž Disadvantages:

ž The camera returns images in a certain
color space
ž You might want to convert to different
color spaces to process it
ž Colour space conversions can take
place between RGB to any other colour
space and vice versa

ž Since cameras usually input images in
rgb
ž We would like to convert these images
into HSV or YCrCb
ž Conversions:
› RGB->HSV
› HSV->RGB
› RGB->YCrCb
› YCrCb->RGB

ž RGB -> HSV

(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa |
Ajusal Sugathan & Utkarsh Sinha
ž HSV RGB YCrCb

ž >>h = rgb2hsv(im)
ž This converts the RGB image to HSV
ž The new colour space components can be
seen using
ž >> imview(h)
ž >> imview(h(:,:,1)) “—HUE—”
ž >> imview(h(:,:,2)) “—Saturation—
”
ž >> imview(h(:,:,3)) “—Value—”

ž >>R = hsv2rgb(im)
ž This converts the HSV image to RGB
ž The new colour space components can
be seen using
ž >> imview(R)
ž >> imview(R(:,:,1)) “—Red—”
ž >> imview(R(:,:,2)) “—Green—”
ž >> imview(R(:,:,3)) “—Blue—”

ž >> Y = rgb2ycbcr(im);
ž This converts the RGB image to YCbCr
ž The new colour space components can be
seen using
ž >> imview(Y)
ž >> imview(Y(:,:,1)) “—Luminance—”
ž >> imview(Y(:,:,2)) “—Differenced
Blue—”
ž >> imview(Y(:,:,3)) “—Differenced
Red—”

ž >> R = ycbcr2rgb(im);
ž This converts the YCbCr image to RGB
ž The new colour space components can
be seen using
ž >> imview(R)
ž >> imview(R(:,:,1)) “—Red—”
ž >> imview(R(:,:,2)) “—Green—”
ž >> imview(R(:,:,3)) “—Blue—”

ž Formulae for conversion are very
complex
ž But the best thing is, you don’t need to
remember these formulae
ž Matlab and OpenCV have built-in
functions for these transformations :-)

ž OpenCV is a collection of many
functions that help in image processing
ž You can use OpenCV in C/C++, .net
languages, Java, Python, etc as well
ž We will only discuss OpenCV in C/C++

ž It is blazingly fast
ž Quite simple to use and learn
ž Has functions for machine learning,
image processing, and GUI creation

ž Download the latest OpenCV package
from
http://sourceforge.net/projects/opencv
/
ž Install the package, and note where
you installed it (like C:Program
FilesOpenCV)

ž Now, we need to tell Microsoft Visual
Studio that we’ve installed OpenCV
ž So, we tell it where to find the OpenCV
header files
ž Start Microsoft Visual Studio 2008

1
2

Type
these
paths
into the
list

ž Right now, Visual Studio knows where
to find the OpenCV include files and
library files
ž Now we create a new project

ž Accept all default settings in the
project
ž You’ll end up with an empty project
with a single file (like Mybot.cpp)
ž Open this file, we’ll write some code
now

ž Add the following at the top of the
code
#include <cv.h>
#include <highgui.h>
ž This piece of code includes necessary
OpenCV functionality

ž Now, we get to the main() function
int main()
{
ž The main function is where for
program execution begins

ž Next, we load an image
IplImage* img = cvLoadImage("C:hello.jpg");
ž The IplImage is a data type, like int,
char, etc

ž Comes built-into OpenCV
ž Any image in OpenCV is stored as an
IplImage thingy
ž It is a “structure”

ž Opens filename and returns it as an
IplImage structure
ž Supported formats:
› Windows bitmaps - BMP, DIB
› JPEG files - JPEG, JPG, JPE
› Portable Network Graphics - PNG
› Portable image format - PBM, PGM, PPM
› Sun rasters - SR, RAS
› TIFF files - TIFF, TIF
› OpenEXR HDR images - EXR
› JPEG 2000 images - jp2

ž Now we show this image in a window
cvNamedWindow("myfirstwindow");
cvShowImage("myfirstwindow", img);
ž This uses some HighGUI functions
(comes along with OpenCV)

ž Creates a window with the caption title
ž This is a HighGUI function
ž You can add controls to each window
as well (track bars, buttons, etc)

ž Shows img in the window with caption
title
ž If no such window exists, nothing
happens

ž Finally, we wait for an input, release
and exit
cvWaitKey(0);
cvReleaseImage(&img);
return 0;
}

ž Waits for time milliseconds, and
returns whatever key is pressed
ž If time=0, waits till eternity
ž Here, we’ve used it to keep the
windows from vanishing immediately

ž Erases img from the RAM
ž Get rid of an image as soon as
possible. RAM is precious J
ž Note that you send the address of the
image (&img) and not just the image
(img)

ž Right now, Visual Studio knows where
OpenCV is
ž But it does not know, whether to use
OpenCV or not
ž We need to tell this explicitly

ž Got errors?
› Check if the syntax is correct
› Copy all DLL files in *OpenCVbin into
C:WindowsSystem32

ž src is the original image
ž dst is the destination
ž code is one of the follow:
› CV_BGR2HSV
› CV_RGB2HSV
› CV_RGB2YCrCb
› CV_HSV2RGB
› CV_<src_space>2<dst_space>

ž src should be a valid image. Or an
error will pop up
ž dst should be a valid image, i.e. you
need a blank image of the same size
ž code should be valid (check the
OpenCV documentation for that)

ž Allocates memory for an image of size
size, with bits bits/pixel and chan
number of channels
ž Used for creating a blank image
ž Use cvSize(width, height) to specify
the size

ž Example:
› IplImage* blankImg =
cvCreateImage(cvSize(640, 480), 8, 3);

ž Wired
› Motor Driving module
› Interface with PC (Parallel/Serial)
ž Wireless
› The Motor-driving module
› The Wireless Receiver Circuit
› The Wireless Transmitter Circuit
› Interface with PC (Parallel/Serial)

ž IC 7805 Voltage Regulator
ž L293D Motor Driver
ž MCT2E Opto-Coupler
ž Parallel Port Male-Connector
ž RF-RX Connector

ž It’s a three terminal linear 5 volt
regulator used to supply the board and
other peripherals
ž Prescribed input voltage to this
component is about 7-9 Volts

ž Voltage fluctuations can be controlled
by using low pass filter capacitors
across output and input
ž Higher input voltage can be applied if
heatsink is provided

ž Used to control Dc and Stepper Motors
ž Uses a H-Bridge which is an electronic
switching circuit that can reverse direction of
current
ž It’s a Dual-H bridge
ž Basically used to convert a low voltage input
into a high voltage output to drive the motor
or any other component
ž Eg: Microcontrollerà Motor Driverà
Motor
(5 Volts) (12
Volts)

ž Different Motor Driver ICs
› L293D
– 600mA Current Rating
– Dual H-bridge (Dc and Stepper Motors)
› L298N
– 1 Amp Current Rating
› L297-L298 (Coupled)
– For stepper motor overdriving
– 2 Ics in parallel
› ULN2003/ULN2803
– 500mA Current Rating
– For unipolar stepper motors

ž Output Current:
› 600 mA
ž Output Voltage
› Wide Range
› 4.5 V – 36 V

ž There are many situations where
signals and data need to be transferred
from one subsystem to another within
a piece of electronics
ž Relays are too bulky as they are
electromechanical in nature and at the
same time give lesser efficiency
ž In these cases an electronic
component called Optocoupler is used

ž They are generally used when the 2
subsystems are at largely different
voltages
ž These use a beam of light to transmit
the signals or data across an electrical
barrier, and achieve excellent isolation

ž In our circuit, Opto-isolator (MCT2E) is
used to ensure electrical isolation
between motors and the PC parallel port
during wired connection
ž The Viz-Board has four such chips to
isolate the four data lines (pin 2, pin 3,
pin 4, pin 5) coming out of the parallel
port

ž Along with the Viz-Board 2 extensions
have been provided i.e
› The Rf Transmitter Module
› The Rf Reciever Module
Transmitt
er
Receive
r

ž Radio frequency modules are used for
data transmission wirelessly at a certain
frequency
ž It sends and receives radio waves of a
particular frequency and a decoder and
encoder IC is provided to encode and
decode this information
ž Wireless transmission takes place at a
particular frequency Eg. 315Mhz
ž Theses modules might be single or dual
frequency

ž Antenna is recommended on both of
them - just connect any piece of 23 cm
long to the Antenna pin
ž The kit has a dual frequency RF
module with frequencies 315/434 Mhz

ž The encoder IC encodes the parallel
port data and sends it to the RF
transmitter module for wireless
transmission
ž They are capable of encoding
information which consists of N
address bits and (12-N) data bits

ž The HT12E Encoder IC has 8 address
bits and 4 data bits
ž A DIP-Switch can be used to set or
unset the address bits A0-A7

A0-A7—Address
Bits
AD8-AD11—Data
Bits

ž The decoder IC decodes the RF
transmitter data and sends it to the
parallel port for wireless transmission
ž They are capable of encoding
information which consists of N
address bits and (12-N) data bits

ž The HT12D Decoder IC has 8 address
bits and 4 data bits
ž A DIP-Switch can be used to set or
unset the address bits A0-A7

A0-A7—Address
Bits
D8-D11—Data
Bits

ž Serial Port
ž Parallel Port
ž USB

ž Data is transferred serially i.e packets
are sent one after the other through a
single port

ž Data is transferred in parallel through
different data pins at the same time
ž Communication is pretty fast
ž Found in old printer ports

25th pin : Ground
2nd-12th pin : I/O lines

ž Parallel port is faster than serial
ž A mass of data can be transmitted at
the same time through parallel ports
ž Though parallel and serial ports are not
found these days in laptops
ž Desktops and old laptops have these
ports

Direct Output from parallel port
Output from motor driver

Camera, object and source positions

Image sampling
and quantization

Continuous image
projected
on an array sensor
Result of image sampling
and quantization

Sampling:
Digitizing the coordinate
values
(spatial resolution)
Quantization:
Digitizing the amplitude
values
(intensity levels)

• 1 bit /pixel
• B bits/pixel
–2B gray levels
–1 byte = 8 bits –> 256
levels
–2 possible values
–2 gray levels -> 0 or 1 (binary
image)

ž All this sampling and quantization puts
in extra noise on the image!
ž Noise can be reduced by
› Using hardware
› Using software: filters

ž Why do we need to enhance images?
ž Why filter images?

ž Large amounts of external
disturbances in real images
ž Due to different factors like changing
lighting and other real-time effects
ž To improve quality of a captured image
to make it easier to process the image

ž First step in most IP applications
ž Used to remove noise in the input
image
ž To remove motion blur from an image
ž Enhancing the edges of an image to
make it appear sharper

ž Generally used types Of Filtering
› Averaging Filter
› Mean Filter
› Median Filter
› Gaussian Smoothing
› Histogram Equalization

ž The Averaging filter is used to sharpen
the images by taking average over a
number of images
ž It eliminates noise by assuming that
different snaps of the same image
have different noise patterns

ž Noise is gaussian in nature i.e follows a
gaussian curve
ž Hence, summing up noises infinite times
approaches zero

ž This is extremely useful for satellites
that take intergalactic photographs
ž The images are extremely faint, and
there is more noise than the image
itself
ž Millions of pictures are taken, and
averaged to get a clear picture

ž The Mean is used to soften an image
by averaging surrounding pixel values
Center pixel =
(22+77+48+150+77+158+0+77+219)/9

ž The center pixel would be changed
from 77 to 92 as that is the mean
value of all surrounding pixels
ž This filter is often used to smooth
images prior to processing
ž It can be used to reduce pixel flicker
due to overhead fluorescent lights

ž This replaces each pixel value by the
median of its neighbors, i.e. the value
such that 50% of the values in the
neighborhood are above, and 50% are
below
ž This can be difficult and costly to
implement due to the need for sorting
of the values
ž However, this method is generally very
good at preserving edges

ž Its performance is particularly good for
removing short noise
ž The median is calculated by first sorting all
the pixel values from the surrounding
neighborhood into numerical order and then
replacing the pixel being considered with the
middle pixel value
ž If the neighborhood under consideration
contains an even number of pixels, the
average of the two middle pixel values is
used

ž Used to `blur' images and remove
detail and noise
ž The effect of Gaussian smoothing is to
blur an image
ž The Gaussian outputs a `weighted
average' of each pixel's neighborhood,
with the average weighted more
towards the value of the central pixels

ž A Gaussian provides gentler smoothing
and preserves edges better than a
similarly sized mean filter
Before
Blurring
After
Blurring

ž It is very useful in contrast enhancement
ž Especially to eliminate noise due to
changing lighting conditions etc
ž Transforms the values in an intensity
image so that the histogram of the output
image approximately matches a specified
histogram

Filters and histograms

ž ‘Imfilter’ function is used for creating
different kinds of filters In MATLAB
ž B = imfilter(A,H,’option’) filters the
multidimensional array A with the
multidimensional filter H
ž The array A can be a nonsparse numeric
array of any class and dimension
ž The result B has the same size and class as A

ž Options in imfilter
ž Convolution is same as correlation except
that the h matrix is inverted before applying
the filter

ž h = ones(5,5) / 25;
ž imsmooth = imfilter(im,h);
ž Here a mean filter is implemented using
the appropriate ‘h’ matrix
ž imshow(im), title('Original Image');
ž figure, imshow(imsmooth),
title('Filtered Image')

ž FSPECIAL is used to create predefined
filters
ž h = FSPECIAL(TYPE);
ž FSPECIAL returns h as a computational
molecule, which is the appropriate
form to use with imfilter

ž The process of adjusting intensity
values can be done automatically by
the histeq function
ž >>im = imread('pout.tif');
ž >>jm = histeq(im);
ž >>imshow(jm)
ž >>figure, imhist(jm,64)

Original Image

Histogram Equalized Image

ž Things aren’t as simple as they were in
Matlab
ž C/C++ needs a bit of syntax and
formalities

ž We’ll try doing the following right now
› Gaussian filter
› Median filter
› Bilateral filter
› Simple blur
› Averaging filter

ž Start Microsoft Visual Studio 2008
ž I assume you have OpenCV installed

#include <cv.h>

#include <cv.h>
int main()
{

#include <cv.h>
int main()
{
IplImage* img = cvLoadImage(“C:noisy.jpg”);

#include <cv.h>
int main()
{
IplImage* img = cvLoadImage(“C:noisy.jpg”);
IplImage* imgBlur = cvCreateImage(cvGetSize(img),
8, 3);
IplImage* imgGaussian = cvCreateImage(cvGetSize
(img), 8, 3);
IplImage* imgMedian = cvCreateImage(cvGetSize
(img), 8, 3);
IplImage* imgBilateral = cvCreateImage(cvGetSize
(img), 8, 3);

cvSmooth(img, imgBlur, CV_BLUR, 3, 3);
cvSmooth(img, imgGaussian, CV_GAUSSIAN, 3, 3);
cvSmooth(img, imgMedian, CV_MEDIAN, 3, 3);
cvSmooth(img, imgBilateral, CV_BILATERAL, 3, 3);

cvNamedWindow(“original”);
cvNamedWindow(“blur”);
cvNamedWindow(“gaussian”);
cvNamedWindow(“median”);
cvNamedWindow(“bilateral”);
cvShowImage(“original”, img);
cvShowImage(“blur”, imgBlur);
cvShowImage(“gaussian”, imgGaussian);
cvShowImage(“median”, imgMedian);
cvShowImage(“bilateral”, imgBilateral);
cvWaitKey(0);
return 0;
}

ž Blur: The plain simple Photoshop blur
ž Gaussian: The best result (preserved
edges and smoothed out noise)
ž Median: Nothing special
ž Bilateral: Got rid of some noise, but
preserved edges to a greater extend

ž Your OpenCV installation comes with
detailed documentation
ž *OpenCVdocsindex.html
ž Scroll down, and you’ll see OpenCV
Reference Manuals

ž Try looking up cvSmooth in the CV
Reference Manual

ž Now try looking up cvEqualizeHist

ž There are no built-in functions for this
ž So, we’ll code it ourselves
ž And this will be a good exercise for
getting better at OpenCV

#include <cv.h>
int main()
{
IplImage* imgRed[25];
IplImage* imgGreen[25];
IplImage* imgBlue[25];
Holds the R, G and B
channels separately
for each of the 25
images

IplImage* imgBlue[25];
for(int i=0;i<25;i++)
{
IplImage* img;
char filename[150];
sprintf(filename, "%d.jpg", (i+1));
img = cvLoadImage(filename);
• Generate the strings “1.jpg”,
“2.jpg”, etc and store them into
filename
• Load the image filename

img = cvLoadImage(filename);
imgRed[i] = cvCreateImage(cvGetSize(img), 8,
1);
imgGreen[i] = cvCreateImage(cvGetSize(img), 8,
1);
imgBlue[i] = cvCreateImage(cvGetSize(img), 8,
1);
• Allocate memory for each component
of image i

imgBlue[i] = cvCreateImage(cvGetSize(img), 8,
1);
cvSplit(img, imgBlue[i], imgGreen[i],
imgRed[i], NULL);
cvReleaseImage(&img);
}
• Split img into constituent channels
• Note the order: B G R
• Release img

ž We created 75 grayscale images: 25
for red, 25 for green and 25 for blues
ž Loaded 25 color images in the loop
ž Split each image, and stored in an
appropriate grayscale image

CvSize imgSize = cvGetSize(imgRed[0]);
IplImage* imgResultRed = cvCreateImage(imgSize, 8,
1);
IplImage* imgResultGreen = cvCreateImage(imgSize,
8, 1);
IplImage* imgResultBlue = cvCreateImage(imgSize,
8, 1);
IplImage* imgResult = cvCreateImage(imgSize, 8,
3);
• This will hold the final, filtered image
• It will be a combination of the
grayscale channels imgResultRed,
imgResultGreen and imgResultBlue

IplImage* imgResult = cvCreateImage(imgSize, 8,
3);
for(int y=0;y<imgSize.height;y++)
{
for(int x=0;x<imgSize.width;x++)
{
• Two loops to take us through the
entire image

for(int x=0;x<imgSize.width;x++)
{
int theSumRed=0;
int theSumGreen=0;
int theSumBlue=0;
{
• To figure out the average, we need to
find the numerator (the sum) over all
25 images

{
theSumRed+=cvGetReal2D(imgRed[i], y,
x);
theSumGreen+=cvGetReal2D(imgGreen[i],
y, x);
theSumBlue+=cvGetReal2D(imgBlue[i], y,
x);
}
• To figure out the average, we need to
find the numerator (the sum) over all
25 images

theSumRed = (float)theSumRed/25.0f;
theSumGreen = (float)theSumGreen/25.0f;
theSumBlue = (float)theSumBlue/25.0f;
cvSetReal2D(imgResultRed, y, x,
theSumRed);
cvSetReal2D(imgResultGreen, y, x,
theSumGreen);
cvSetReal2D(imgResultBlue, y, x,
theSumBlue);
}
}
• Once we have the sum, we divide by
25 and set the appropriate pixels

cvMerge(imgResultBlue, imgResultGreen,
imgResultRed, NULL, imgResult);
cvNamedWindow("averaged");
cvShowImage("averaged", imgResult);
cvWaitKey(0);
return 0;
}
• Merge the three channels, and
display the image

ž cvLoadImage always loads as BGR
ž cvSplit to get the individual channels
ž cvMerge to combine individual
channels into a color image

ž IplImage to store any image in
OpenCV
ž cvCreateImage to allocate memory
ž cvReleaseImage to erase an image
from the RAM

ž cvWaitKey to get a keypress within
certain milliseconds
ž cvNamedWindow to create a window
ž cvShowImage to show an image in a
window

ž cvGetReal2D to get value at a pixel in
grayscale images
ž cvSetReal2D to set the value at a pixel
ž CvSize to store an image’s size

ž you can always refer to the OpenCV
documentation

ž The process of extracting image
components that are useful in
representation of image for some
particular purpose
ž Basic morphological operations are:
› Dilation
› Erosion

ž The operation that grows or thickens
objects in a binary image
ž The specific manner of thickening is
controlled by a shape referred to as
“structuring element”

Structuring
Element
Binary
Image

Dilated
Image

ž Erosion shrink or thins objects in a
binary image
ž The manner of shrinkage is controlled
by the structuring element

Eroded
Image

ž In practical image processing dilation
and erosion are performed in various
combinations
ž An image can undergo a series for
diltions and erosion using the same or
different structuring element

ž In practical image processing dilation
and erosion are performed in various
combinations
ž An image can undergo a series for
diltions and erosion using the same or
different structuring element
ž Two Common Kinds:
› Morphological Opening
› Morphological Closing

ž It is basically one erosion followed by
one dilation by the same structuring
element
ž They are used to smooth object
contours, break thin connections and
remove thin protrusions

A—Image
B—Structuring element

ž It is basically one dilation followed by one
erosion by the same structuring element
ž They are used to smooth object contours
like opening
ž But unlike opening they generally join
narrow breaks, fill long thin gulfs and fills
holes smaller than the structuring
element

ž Used to generate a structuring
element
ž >>se=strel(shape,parameters)

ž Dilation in matlab is done using the
following command:
ž >>bw2=imdilate(bw,st)
ž Bw = Original image
ž St = Structuring element

ž Erosion in matlab is done using the
following command:
ž >>bw2=imerode(bw,st)

ž Opening in matlab is done using the
following command:
ž >>bw2=imopen(bw,st)

ž Closing in matlab is done using the
following command:
ž >>bw2=imclose(bw,st)

ž cvErode(src, dst)
ž cvDilate(src, dst)
ž Opening & closing: use the appropriate
sequence

ž By default, OpenCV uses the zero
structuring element (all are zeros)
ž You can explicitly specify your
structuring element as well
ž Check the OpenCV Documentation for
more information

ž Computers can manipulate images
very efficiently
ž But, comprehending an image with
millions of colors is tough
ž Solution: Figure out interesting
regions, and process them

ž Each pixel is checked for its value
ž If it lies within a range, it is marked as
“interesting” (or made white)
ž Otherwise, it’s made black
ž Figuring out the range depends on
lighting, color, texture, etc

ž Demo thresholdRGB
ž Demo thresholdHSV

ž MATLAB provides a facility to execute
multiple command statements with a
single command. This is done by
writing a .m file
ž Goto File > New > M-file
ž For example, the graythresh function
can be manually written as a m-file as:

ž Observe that, comments (in green) can
be written after the symbol ‘%’. A
commented statement is not considered
for execution
ž M-files become a very handy utility for
writing lengthy programs and can be
saved and edited, as and when required
ž We shall now see, how to define your own
functions in MATLAB.

ž Functions help in writing organized
code with minimum repetition of logic
ž Instead of rewriting the instruction set
every time, you can define a function
ž Syntax:
ž Create an m-file and the top most
statement of the file should be the
function header
ž function [return values] = function-
name(arguments)

ž The inbuilt graythresh function in
matlab is used for thresholding of
grayscale images
ž It uses the Otsu’s Method Of
thresholding
ž A sample thresholding opreation has been
shown in the next slide

Image thresholded for
the colour blue

The Real thing J

ž Thresholding of a grayscale image can
be done in MATLAB using the following
commands:
ž >> level=graythresh(imGRAY);
ž >> imBW = im2bw(imGRAY,level);
ž >> imview(imBW);

ž The graythresh command basically gives
an idea as to what exactly the threshold
value should be
ž Graythresh returns a value that lies in the
range 0-1
ž This gives the level of threshold which is
obtained by a complex method called the
Otsu’s Method of Thresholding

ž This level can be converted into pixel
value by multiplying by 255
ž Lets say, level=.4
ž Then threshold value for the grayscale
image is:
ž 0.4 x 255 =102

ž What this indicates is that for the given
image the values below 102 have to
be converted to 0 and values from
103-255 to the value 1
ž Conversion from grayscale to binary
image is done using the function:
ž >>imBW = im2bw(imGRAY,level);

ž Here level is the threshold level
obtained from graythresh function
ž This function converts pixel intensities
between 0 to level to zero intensity
(black) and between level+1 to 255 to
maximum (white)

ž In order to threshold an RGB colour
image using the graythresh function, the
following have to be done:
› Conversion of the RGB image into its 3
grayscale components
› Subtracting each of these components from
the other 2 to get the pure colour intensities
› Finding level for each of the grayscale using
graythresh
› Thresholding the image using imbw and the
level

ž Commands:
ž Im=Imread(‘rgb.jpg’);
ž R = im(:,:,1); --Red
ž G = im(:,:,2); --Green
ž B = im(:,:,3); --Blue

ž Ronly=R-B-G; --Pure RED
ž Gonly=G-R-B; --Pure GREEN
ž Bonly=B-G-R; --Pure BLUE

ž Levelr=graythresh(Ronly);
ž Levelg=graythresh(Gonly);
ž Levelb=graythresh(Bonly);

ž Rthresh=im2bw (Ronly,levelR);
ž Gthresh=im2bw(Gonly,levelG);
ž Bthresh=im2bw(Bonly,levelB);

ž Using a manually designed thresh_tool
function to adjust the levels as
required
ž To get a feel of how levels vary

s=size(im);
temp=im;
thresh=128;
for i=1:s(1,1)
for j=1:s(1,2)
if temp(i,j)<thresh
temp(i,j)=0;
else
temp(i,j)=255;
end
end
end
imview(temp);

ž Splitting of HSV image into
components
ž Using the Hue channel and
thresholding it for different values
ž Since the hue value of a single colour
is constant it is relatively simple to
threshold and gives better accuracy

function [temp] = ht(im,level1,level2)
s=size(im);
temp=im;
for i=1:s(1,1)
for j=1:s(1,2)
if (temp(i,j)<level2 & temp(i,j)>level1)
temp(i,j)=1;
else
temp(i,j)=0;
end
end
end
imview(temp);

ž To this function we give the input
arguments as the upper and lower
bounds of the threshold levels
ž These levels can be obtained by
having a look at the range of hue
values for the particular colour

Now that you know the
basics

ž cvThreshold(src, dst, threshold, max,
type)
ž type:
› CV_THRESH_BINARY
› CV_THRESH_BINARY_INV
› And several others (check documentation)

ž cvInRangeS(src, scalarLower,
scalarUpper, dst);
ž scalarLower = cvScalar(chan1, chan2,
chan3, chan4);
ž scalarUpper = cvScalar(chan1, chan2,
chan3, chan4);

ž Ultra basics: motors, drives, etc
ž Digital image representation
ž Color spaces
ž Inter-conversion of color spaces
ž Electronics
ž Filtering
ž Thresholding
ž Morphology

ž After thresholding, we get a binary
image
ž We want useable information like
centers, outlines, etc
ž There geometrical properties can be
found using many methods. We’ll talk
about moments and contours only.

ž Moments are a mathematical concept
ž ∑ ∑intensity*xxorder*yyorder

ž Consider xorder=0 and yorder=0 for a
binary image
ž So you’re just summing up pixel values
ž This means, you’re calculating the area
of the white pixels

ž Now consider xorder=1 and yorder=0
for a binary image
ž You sum only those x which are white
ž So you’re calculating the numerator of
an average

ž The number of points where the pixel
is white is the area of the image
ž So, dividing this particular moment
(xorder=1, yorder=0) by the earlier
example (xorder=0, yorder=0) gives
the average x
ž This is the x coordinate of the centroid
of the blob

ž Similarly, for xorder=0 and yorder=1,
you’ll get the y coordinate

ž The order of a moment =
xorder+yorder
ž So, the area is a zero order moment
ž The centroid coordinate = a first order
moment / the zero order moment

ž There are entire books written on this
topic
ž You can find complex geometrical
properties, like the eccentricity of an
ellipse, radius of curvature of objects,
etc
ž Also check for Hu invariants if you’re
interested

Centroid Area etc

ž These are pixels of an image that are
conencted to each other forming
separate blobs in an image
ž They can be seperated out and labelled

ž >>L = bwlabel(BW,n)
ž Returns a matrix L, of the same size as
BW, containing labels for the connected
objects in BW
ž n can have a value of either 4 or 8, where
4 specifies 4-connected objects and 8
specifies 8-connected objects; if the
argument is omitted, it defaults to 8

ž STATS = regionprops(L,properties)
ž Measures a set of properties for each
labeled region in the label matrix L
ž The set of elements of L equal to 1
corresponds to region 1; the set of
elements of L equal to 2 corresponds
to region 2; and so on

ž 'Area'– The actual number of pixels in the
region
ž 'Centroid'-- The center of mass of the region.
Note that the first element of Centroid is the
horizontal coordinate (or x-coordinate) of the
center of mass, and the second element is
the vertical coordinate (or y-coordinate)
ž 'Orientation' -- Scalar; the angle (in degrees)
between the x-axis and the major axis of the
ellipse that has the same second-moments
as the region. This property is supported
only for 2-D input label matrices

ž BW = imread('text.png');
ž L = bwlabel(BW);
ž stats = regionprops(L,'all');

ž Label into an RGB image for better
vizualization
ž RGB = label2rgb(L)

ž Binary area open remove small objects
ž BW2 = bwareaopen(BW,P)
ž Removes from a binary image all
connected components (objects) that
have fewer than P pixels, producing
another binary image, BW2.

ž OpenCV supports functions to calculate
moments upto order 3
CvMoments *moments = (CvMoments*)malloc(sizeof
(CvMoments));
cvMoments(img, moments, 1);

ž cvGetSpatialMoment(moments, xorder,
yorder)
ž cvGetCentralMoment(moments, xorder,
yorder)
ž Central = spatial/area

ž Example

ž For robotics purposes, moments are
fine till have one single object
ž If we have multiple objects in the same
binary image

ž You can think of contours as an
approximation of a binary image

ž You get polygonal approximation of
each connected area

ž The output you get for the previous
binary image is:
› Four “chains” of points
› Each chain can have any number of points
› In our case, each chain has four points

Contour plotting

ž Contour plot of an image im can be made
in MATLAB using the command:
ž im = imread(‘img.jpg');
ž imcontour(im,level)
ž Level=number of equally spaced contour
levels
ž if level is not given it will choose
automatically

ž OpenCV linked lists to store the
“chains”
ž We’ll see some code to find out the
squares in the thresholded image you
saw

CvSeq* contours;
CvSeq* result;
CvMemStorage *storage = cvCreateMemStorage(0);
• The chains are stored in contours
• result is a temporary variable
• storage is for temporary memory allocation

cvFindContours(img, storage, &contours, sizeof(CvContour),
CV_RETR_LIST, CV_CHAIN_APPROX_SIMPLE, cvPoint(0,0));
• img is a grayscale thresholded image
• storage is for temporary storage
• All chains found would be stored in the contours sequence
• The rest of the parameters are usually kept at these values
• Check the OpenCV documentation for details information about the
last four variables

The RoboCV Workshop

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to The RoboCV Workshop

Similar to The RoboCV Workshop (20)

Recently uploaded

Recently uploaded (20)

The RoboCV Workshop