1. S W I S S G E R M A N U N I V E R S I T Y
DESIGNING A SMART CAR DATA LOGGING AND TRACKING
WITH VISION
A Thesis Presentation
By
Willy Haliim
In Partial Fulfillment of the Requirements for
BACHELOR’S DEGREE
IN
MECHATRONICS ENGINEERING
SWISS GERMAN UNIVERSITY
EduTown BSDCity
Tangerang – 15339
INDONESIA

2. S W I S S G E R M A N U N I V E R S I T Y
15/07/14 2
PRESENTATION OVERVIEW
• Background Problem
• Thesis Purpose
• Methodology
• Results
• Conclusion
• Discussion / Further Recommendation
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology

3. S W I S S G E R M A N U N I V E R S I T Y
15/07/14 3
JOINT PROJECT OF THREE STUDENTS
1. Building a Data Logger as Part of Intelligent Vehicle
System Using 32-bit ARM Cortex-M3 with Cubieboard and
Android Phone Interfacing. (Richard‘s Part)
2. Designing a Smart Car Data Logging and Tracking
With Vision. (Author‘s Part)
3. Designing an Inter-Cars Safety Communication System
Based on Android Smartphone. (Rendi‘s Part)
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology

4. S W I S S G E R M A N U N I V E R S I T Y
4
BACKGROUND
15/07/14
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology
Accident Prevention Accident Handling

5. S W I S S G E R M A N U N I V E R S I T Y
OVERALL SYSTEM IDEA
15/07/14
Smart Mobility with Connected Cooperative Cars System
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology

6. S W I S S G E R M A N U N I V E R S I T Y
6
THESIS PURPOSE
15/07/14
Author’s Scope:
1. Prevent accident by developing an early warning range detection
system using stereovision system
2. Communicate with other subsystem to create a whole working smart
mobility conected cooperative cars.
3. Use Dual Core ARM Cortex-A7 Mini PC and Linux
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology

7. S W I S S G E R M A N U N I V E R S I T Y
7
PROBLEM DEFINITION
1. How to detect obstacle‘s range in front of a stereo camera using
stereovision system.
2. How to record the car‘s journey using one the stereo camera.
3. Which Communication Method will be used to synchronize the
Cubieboard 2 with the sensor and the server subsystems.
4. How to explore the exposed I/O pin on Cubieboard 2 to minimize
hardware requirements.
5. The processing speed is limited by the specification of
Cubieboard 2. The Focus is on Functionality.
15/07/14
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology

8. S W I S S G E R M A N U N I V E R S I T Y
15/07/14
OVERALL SYSTEM OVERVIEW
(Author’s Scope)
8
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology

9. S W I S S G E R M A N U N I V E R S I T Y
SYSTEM OVERVIEW
STEREOVISION
RANGE CALCULATION
PROCESS
ESTIMATED
OBJECT’S RANGE
INPUT PROCESSING OUTPUT
15/07/14 9
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology
Minoru 3D Camera Cubieboard 2
IMAGES
UI PUSHBUTTON

10. S W I S S G E R M A N U N I V E R S I T Y
15/07/14 10
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology
CUBIEBOARD 2. DUAL CORE 1 GHz. 1 GB RAM

11. S W I S S G E R M A N U N I V E R S I T Y
19/02/10 [Your Thesis Title] – [Your Name] – [Your Faculty]
INSIDE THE ARM CORTEX A7 PROCESSOR

12. S W I S S G E R M A N U N I V E R S I T Y
ARM PROCESSORS
15/07/14 12
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology
OS Application Based.
Programmable Using C++
Low Power Consumption
Low Cost

13. S W I S S G E R M A N U N I V E R S I T Y
THESIS METHODOLOGY
15/07/14 13
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology

14. S W I S S G E R M A N U N I V E R S I T Y
INSTALLING WORKSPACE ENVIRONMENT
15/07/14
CUBIAN OS REMOTE TO SMARTPHONE
14
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology

15. S W I S S G E R M A N U N I V E R S I T Y
STEREOVISION METHOD 1 - DEPTH IMAGE
15/07/14
DISPARITYRECTIFICATION
CAMERA
CALIBRATION
15
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology
STEREO MATCHING

16. S W I S S G E R M A N U N I V E R S I T Y
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology
15/07/14
|l-r|
α
α
β
β
16
STEREOVISION METHOD 2 – RANGE CALCULATION
b

17. S W I S S G E R M A N U N I V E R S I T Y
17
PROGRAM FLOWCHART - RANGE CALCULATION
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology
15/07/14
OBJECT DETECTING EARLY WARNING SYSTEM CAPTURE AND SEND

18. S W I S S G E R M A N U N I V E R S I T Y
OBJECT DETECTING STEPS – METHOD 2
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology
15/07/14
3. Color Filter
Erode Dilate
4. Find & Draw
Contours
5. Calculate
the COG
1. RGB Image
(Original)
2. Convert to
HSV
18
Detects yellow object
Centroid object detected
inRange

19. S W I S S G E R M A N U N I V E R S I T Y
A B C
EARLY WARNING RANGE DETECTION SYSTEM
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology
15/07/14
RANGE LED INDICATOR DISPLAY
100 – 150 CM GREEN SAFE
50 – 99 CM YELLOW -
BELOW 50 CM RED AND BUZZER WARNING!
19
R1 R2 R3

20. S W I S S G E R M A N U N I V E R S I T Y
SERIAL
DATA COMMUNICATION
RECEIVED AND SENT
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology
15/07/14
FROM
FLYMAPLE BOARD
(SENSOR)
TO
ANDROID PHONE
(SERVER)
FTP
20
Location data
Orientation Data
Accident Log
(Trigger)
Location data
Accident Log
(Trigger)
Image Captured

21. S W I S S G E R M A N U N I V E R S I T Y
21
RESULTS – OVERVIEW DESIGN
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology
15/07/14
Stereo Camera
Cubieboard 2
Remote Desktop
Flymaple
(Sensor)
Wifi dongle
Early warning
system

22. S W I S S G E R M A N U N I V E R S I T Y
22
RESULTS – DEPTH IMAGE
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology
15/07/14
CAMERA ON FIXED POSITION.
OBJECT MOVED EVERY 10 CM.

23. S W I S S G E R M A N U N I V E R S I T Y
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology
15/07/14
Distance
(cm)
Average
Grayscale
Value
10 -
20 152
30 111
40 86
50 65
60 52
70 44
80 37
90 31
100 26
110 22
120 19
130 16
140 13
150 11
23
RESULTS – DEPTH IMAGE

24. S W I S S G E R M A N U N I V E R S I T Y
24
RESULTS – DEPTH IMAGE
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology
15/07/14
CUBIAN (320 X 480) WINDOWS 7 (640X480)

25. S W I S S G E R M A N U N I V E R S I T Y
RESULTS – RANGE CALCULATION
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology
15/07/14 25
Program UI and
Display Frame

26. S W I S S G E R M A N U N I V E R S I T Y
RESULTS – RANGE CALCULATION
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology
15/07/14 26
Object on fixed position. Camera moved every 10 cm.

27. S W I S S G E R M A N U N I V E R S I T Y
15/07/14
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology 27
Real
Range(mm)
Avg Disparity
(Pix)
Average(mm) Error(mm) Error(%)
100 56 625 -525 84
200 79 443 -243 55
300 86 404 -104 26
400 75 464 -64 14
500 63 547 -47 9
600 55 631 -31 5
700 49 705 -5 1
800 45 774 26 3
900 41 841 59 7
1000 39 897 103 11
1100 36 957 143 15
1200 35 1001 199 20
1300 34 1031 269 26
1400 32 1095 305 28
1500 30 1195 305 30
1600 29 1201 399 33
= 3.3%

28. S W I S S G E R M A N U N I V E R S I T Y
15/07/14
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology 28
= 3.3%

29. S W I S S G E R M A N U N I V E R S I T Y
15/07/14
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology 29
= 3.3%
The closer you look the bigger the disparity. The less you see

30. S W I S S G E R M A N U N I V E R S I T Y
A B C
EARLY WARNING RANGE DETECTION SYSTEM
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology
15/07/14
FAR MEDIUM CLOSE
30

31. S W I S S G E R M A N U N I V E R S I T Y
DATA COMMUNICATION
RECEIVED AND SENT
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology
15/07/14
FROM FLYMAPLE BOARD DATA ON ANDROID SERVER
31
song.mp3

32. S W I S S G E R M A N U N I V E R S I T Y
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology
15/07/14
IMPROVEMENTS
32
Without multithreaded
low FPS
With multithreaded
higher FPS

33. S W I S S G E R M A N U N I V E R S I T Y
33
CONCLUSION
1. Obstacle in front of camera successfully detected with effective
range 40 – 100 cm and error around 1 – 10 % .
2. Wired and Wireless Communication between Flymaple board
sensor and Android server are successfully attempted.
3. I/O pins explored and it is useful for input output sensors, like
capturing button and early warning range detection system.
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology
15/07/14

34. S W I S S G E R M A N U N I V E R S I T Y
34
DISCUSSION / FURTHER RECOMMENDATION
1. Use ARM Cortex-A7 processor with better
specification to provide faster calculation and gives
better frame rate result.
2. Stereo Camera with higher resolution, bigger baseline
and focal length to acquire further distance detection.
3. Better algorithm on stereovision programming.
4. Improve stereovision range detection by using another
stereovision calculation method.
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology
15/07/14

35. S W I S S G E R M A N U N I V E R S I T Y
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology
15/07/14
PRESENTATION RESULTS DEMO
35
EARLY RANGE DETECTION WARNING SYSTEM

36. S W I S S G E R M A N U N I V E R S I T Y
Designing a Smart Car Data Logging and Tracking with Vision – Willy Haliim –
Engineering and Information Technology
15/07/14
PRESENTATION RESULTS DEMO
36
FTP COMMUNICATION