2. Prepared By
Ashifur Rahman (1312029)
ASM Arman (1312038)
Bhuvon Costa (1312045)
Under the Supervision of
Dr. Mashud Karim
Professor
Department of Naval Architecture and
Marine Engineering
Bangladesh University of Engineering and
Technology.
2
4. Objective
The objective of this research is to –
Make an automated port loading-unloading system
based on the free space available on port using
machine learning
Categorizing port containers based on the products
inside the containers using machine learning
4
5. Motivation
Roughly ninety percent of the world's goods are
transported by sea with over seventy per- cent as
containerized cargo. This containers are handled
through sea ports.
Most of the containers in our only sea port is handled
manually. This consumes a lot of human power and time
and is very costly.
5
6. Motivation
6
Automation of port cargo handling will result in reduction
of human power, less consumption of time and will be cost
effective.
Using machine learning will open door to many other
possibilities like automated security system in port area.
Machine learning can be used to detect less fuel
consumption, low carbon emission routes can be detcted.
7. Methodology
We have used Tensorflow Object Detection API
developed by Google to achieve our objective.
To detect port containers which are custom object for this
API, we had to train our machine.
For training purpose we have built container model which
are made of wood from carpentry shop.
7
9. Methodology
9
After building container model, we have taken around 500
photos of our model form different angle.
We have then trained our machine to detect container
model.
Our trained machine then become able to detect custom
container object from live video feed.
11. Detection API Used
Creating accurate machine learning models capable of
localizing and identifying multiple objects in a single image
remains a core challenge in computer vision.
The TensorFlow Object Detection API is an open source
framework built on top of TensorFlow that makes it easy to
construct, train and deploy object detection models.
11
12. Python Packages Used
12
We have used several python packages to achieve our
objective. They are-
1.Tensorflow to detect object
2.Numpy for making image array
3.Matplotlib for plotting data
4.Opencv to capture live video feed
13. Hardware Used to Build Model
We have used Arduino UNO to build our model
13
15. 15
For Training Process we have taken around 500 photos of
our model from different angle.
After that we labeled the container in the image to make it
detectable for TensorFlow. For this purpose we used
tzutalin/levelimg.
Training Process
17. Training Process Contd.
17
After leveling our image, Levelimg package make a .xml file
containing matching annotation information for each
image.
We have splitted our image files along with their respective
.xml files in two folders: train and test.10% of images with
their respective xml files are put into test folder and the rest
90% are put into train folder.
18. Training Process Contd.
18
We have used ssd_mobilenet_v1_coco_11_06_2017 as a
training model because it is faster than other model. so, it
takes less time to train model.
After, configuring all our files, we started to train our
machine to detect our container model.
19. Training Process Contd.
19
We have converted our .xml files to two.csv file
train_levels.csv and test_levels.csv using xml_to_csv.py
file.
After converting .xml to .csv, we have converted .csv files
into .tfr file using generate_tfrecord.py. Produced
train.record was used to train the machine
20. Training Process Contd.
20
We have created object-detection.pbtxt file to store the
obeject’s classification and id, this will help the detection
API to give the tracked object a classification.
We have taken 225 iteration, after the iteration the
detection API created a checkpoint file model.ckpt-
225.meta. This checkpoint file recorded the learning data
up to that iteration.
21. Training Process Contd.
21
As we don’t have a dedicated GPU and our machine only
run on a CPU. Our machine is not trained properly.
After training process our total loss was below 2. That
means our machine can detect within the range of 98%
accuracy.
For a decent training, it would take 10,000 iteration but we
took only around 200 iteration.
26. Detecting Container from Live Video
26
One of the biggest challenge of our research was detecting
container from live video feed.
We have used OpenCV for this purpose. OpenCV captured
video feed from our default web cam and then pass it to our
trained machine to detect container from it.
27. Detecting Container from Live Video
27
As we only run on CPU, detection process was quite slow.
Another challenge of our research was calculating the
position of detected containers. For this purpose we have
subdivided our screen into two grid and developed an
algorithm that will send an specific message when a
container is detected on a grid.
30. Model Making Process
We have used several materials to build our model.
The frame of model is made of steel.
We have used Arduino UNO as an interface between the model and
our object detection algorithm.
We have used two servo motor to make our model functional.
For the time being, we are using USB web cam of our machine.
30
31. Model Making Process
After making the model we have started to make an
interface which will run the model after the detection API
detect a container.
For this we have used Python Pyserial package to enable
the objection detection API to communicate with arduino.
31
32. Model Making Process
Pyserial package sends a specific signal based on the
location where the container is detected to the serial port
which enables the arduino to rotate the servo motor to a
specific rotation number which in turns send the crane to
the specific location of the conatiner.
Unlike the last demonstration, we have decided to use static
image instead of video to detect our containers.
32
33. Model Making Process
33
Camera will take an image of the container area or the ship
which is carrying the containers and the image will be then
rendered by object detection API to detect all the containers
on that ship.
After detecting all the containers, the detection API will
start to send specific signal to the port and arduino will
enables the mechanism to handle the container via crane.
38. Result
38
Iteration Image No Orientation Loss Confidence
245 465 Different
Angle
1.52 98.48
225 235 Top View 1.22 98.78
220 345 Top View 1.16 98.84
39. Limitations
39
Our accuracy level was not 100% due to incapability of
machine
As machine was trained with high definition images so it
can not detect object from low definition video
Our model have not any active handling system
A feasibility calculation for real word automated port
handling system was not provided
