Slidedeck for my presentation on building a self-serving rc car as presented at Devoxx UK 2018

1. Tim van Eijndhoven
@TimvEijndhoven
Building a self-driving RC car

2. @bjschrijver@TimvEijndhoven
Let’s meet
Tim van Eijndhoven

3. @bjschrijver@TimvEijndhoven
Let’s meet
Bert Jan Schrijver

4. @TimvEijndhoven
How
it all started…

5. @TimvEijndhoven
RoboRace challenge

6. @TimvEijndhoven
The rules
• 4 teams
• Each team gets:
• RC car kit
• Fixed budget (150 euro)
• Three races:
1. drag race (start, drive, stop)
2. race track (race 1 + corners)
3. destruction derby ;-) (race 2 with multiple cars on same track)

7. @TimvEijndhoven
The kit

8. @TimvEijndhoven
The brain

9. @TimvEijndhoven
The platform

10. @TimvEijndhoven
Vert.x
• Toolkit for building reactive applications on the JVM
• Event-driven, non-blocking
• General purpose application framework
• Why Vert.x for our project?
• Lightweight, fast
• Polyglot
• Distributed eventbus (browser included!)
• Building microservices with Vert.x: https://youtu.be/yLg-LPSRjho

11. @TimvEijndhoven
Connecting the dots

12. @TimvEijndhoven
GPIO
• General purpose input/output

13. @TimvEijndhoven
Speaking the same language

14. @TimvEijndhoven
PWM
• Pulse-width modulation, repeating pulses of variable width
• ServoBlaster: https://github.com/richardghirst/PiBits/tree/
master/ServoBlaster

15. @TimvEijndhoven
Staying safe

16. @TimvEijndhoven
Getting visual

17. @TimvEijndhoven
RPi Cam Web Interface
• Web interface for the Raspberry Pi Camera module
• Video streaming
• Web interface to configure video settings
https://github.com/silvanmelchior/RPi_Cam_Web_Interface

18. @TimvEijndhoven
Ready for the road

19. @TimvEijndhoven
Putting stuff together

20. @TimvEijndhoven
Learning
to
drive

21. @TimvEijndhoven
OpenCV
• Open source computer vision and machine learning library
written in C++ with Java interfaces
• Optimised algorithms for computer vision
• Most widely used computer vision library
• Many resources available

22. @TimvEijndhoven
1: Canny edge detection 2: Probabilistic Hough Line Transform
3: Filter group and average lines 4: Determine lane center

23. @TimvEijndhoven
Show
m
e
som
e
code

24. @TimvEijndhoven
Boars Nest (Laptop) Duke Farm (Car)
Daisy
Luke
Bo
Servoblaster
Boss
Hogg
Roscoe
Flash
GeneralLee
Heartbeat
Manual inputs
Steering / Speed
instructions
Start/Stop
autopilot
Image feed
Angle &
Distance
Intermediate
CV images

25. @TimvEijndhoven
Expectation

26. @TimvEijndhoven
Reality

27. @TimvEijndhoven

28. @TimvEijndhoven

29. @TimvEijndhoven

30. @TimvEijndhoven

31. @TimvEijndhoven

32. @TimvEijndhoven
Let’s race!

33. @TimvEijndhoven

34. @TimvEijndhoven

35. @TimvEijndhoven
In
the
real w
orld

36. @TimvEijndhoven

37. @TimvEijndhoven

38. @TimvEijndhoven

39. @TimvEijndhoven

40. @TimvEijndhoven

41. @TimvEijndhoven
Examples of safety measures
• Geographic area limitation
• Strictly defined conditions
• Redundancy
• Multiple sensors
• Isolating control systems
• Human intervention fallback

42. @TimvEijndhoven
Ethics

43. @TimvEijndhoven
http://moralmachine.mit.edu

44. @TimvEijndhoven
http://moralmachine.mit.edu

45. @TimvEijndhoven
“In hazardous situations that prove to be
unavoidable, despite all technological
precautions being taken, the protection
of human life enjoys top priority in a
balancing of legally protected interests.”

46. @TimvEijndhoven
“In the event of unavoidable accident
situations, any distinction based on
personal features (age, gender, physical
or mental constitution) is strictly
prohibited.”

47. @TimvEijndhoven
Future improvements
• Improve detection and navigation of curved lanes
• Optimize line detection algorithm using sliding window
• Automatic (re)calibration of:
• Video parameters
• OpenCV parameters
• Add Flight recorder to enable replay and debug
• Control servos directly from Java
• Use AI for navigation
• Optimize for running everything on the car

48. Questions?
@TimvEijndhoven

49. Thanks for your time.
Got feedback? Tweet it!
All pictures belong
to their respective
authors
https://github.com/rc-dukes
@TimvEijndhoven