Possible COLREGs Failures under Digital Helmsman of Autonomous Ships

1. Introduction Ship Intelligence Framework Ship Collision Avoidance Conclusions
Possible COLREGs Failures under Digital Helmsman
of Autonomous Ships
Lokukaluge Prasad Perera & Bjorn-Morten Batalden1
1Department of Technology and Safety
UiT The Arctic University of Norway
Tromso
Autonomous Ship Technology Symposium
June, 2019
Amsterdam, The Netherlands
UiT The Arctic University of Norway 1 / 25

2. Introduction Ship Intelligence Framework Ship Collision Avoidance Conclusions
Outline
1 Introduction
Transportation Systems
Ship Maneuvering
System Intelligence
Digital Helmsman
2 Ship Intelligence Framework
Ship Intelligence Framework
3 Ship Collision Avoidance
Collision Avoidance Features
Collision Risk Estimation
Collision Avoidance
COLREGs
4 Conclusions
The End
UiT The Arctic University of Norway 2 / 25

3. Introduction Ship Intelligence Framework Ship Collision Avoidance Conclusions
Transportation Systems
Autonomous Navigation
Autonomous navigation will play an important role in future transportation
systems.
The technologies required for autonomous navigation in land transportation
systems, i.e. self-driving cars such as Tesla, Uber, and Waymo, are in a mature
phase and the environment is structured, i.e. well-defined roads and
communication networks.
The required technological advancements for autonomous transportation
systems in an unstructured environment are subject to more challenging
navigation constraints.
Not only the required technologies for maritime transportation systems can
be more complex and still in a development phase, but also the present
infrastructure is inadequate, in general.
A considerable amount of infrastructure and technology challenges has been
encountered by maritime transportation systems in relation to autonomous
navigation.
The required fundamental technologies to support future maritime
transportation systems under autonomous navigation should be developed.
This requires an understating of the challenges associated with ship navigation
and finding appropriate solutions.
UiT The Arctic University of Norway 3 / 25

4. Introduction Ship Intelligence Framework Ship Collision Avoidance Conclusions
Ship Maneuvering
Vessel Controllability Problem
Vessel position (i.e. the centre of vessel
rotation) : P(x(t), y(t))
Course-speed vector : V(t)
Heading (surge) vector : u(t)
Sway velocity : v(t)
Drift angle : β(t)
Ship manoeuvring consists of complex rigid body motions.
This is due to large bandwidth of nonlinear hydrodynamic and wind force and
moment interactions between environment and the vessel hull and
superstructure, which often generate unexpected and undesirable ship
motions.
A vessel can have a heading vector that deviates from the course-speed
vector, resulting in a drift angle.
Since vessels are not navigating in fully-defined ship routes, one vessel can
encounter other vessels in its vicinity with various course-speed and heading
vectors.
Ship navigators should be aware of such encounters associated with higher
collision risk.
UiT The Arctic University of Norway 4 / 25

5. Introduction Ship Intelligence Framework Ship Collision Avoidance Conclusions
Ship Maneuvering
Vessel Controllability Problem (cont.)
Vessel position (i.e. the centre of vessel
rotation) : P(x(t), y(t))
Course-speed vector : V(t)
Heading (surge) vector : u(t)
Sway velocity : v(t)
Drift angle : β(t)
Full controllability of vessels with rudder and propeller actuators is not
possible and is especially demanding under rough weather conditions.
Vessels present sway-yaw manoeuvring interactions and are thus
under-actuated systems with heavy inertia.
The course-speed vector cannot be measured or estimated with enough
accuracy, i.e. not enough sensor measurements.
The centre of vessel rotation can also change due to the environmental
loads.
Not only the respective navigation vectors but also their positions can change
without any requests.
Vessels are considered as slow response systems with considerable
time-delays in response to discrete control requests.
UiT The Arctic University of Norway 5 / 25

6. Introduction Ship Intelligence Framework Ship Collision Avoidance Conclusions
System Intelligence
Classical Mechanics 2 AI
Various advanced controllers based on classical mechanics have been
introduced by the research community to address this ship controllability
problem , however the outcomes are still not satisfactory.
The main reason is that these mathematical models may not adequately
capture the complexities in ship motions; therefore controller robustness
and/or stability cannot be preserved during ship manoeuvres.
The controller inputs, i.e. rudder and propeller control inputs, are not
continuous and the controller outputs, i.e. heading and course-speed vectors,
may not have adequate accuracy and/or associated time-delays, hence
controller performance can be further degraded.
Though conventional ship auto-pilot systems are based on similar approaches,
such systems may not able to handle complex navigational constraints.
Ocean going vessels are still navigated by humans, especially in cluttered
navigation zones and rough weather, using their knowledge and experiences to
overcome complex vessel motions.
Our aim is to overcome these issues in ship navigation by introducing Artificial
Intelligence (AI) into the ship controllability problem.
That has categorized as Cloning Ship Navigator Behavior.
UiT The Arctic University of Norway 6 / 25

7. Introduction Ship Intelligence Framework Ship Collision Avoidance Conclusions
System Intelligence
Self-driving Cars 2 Autonomous Ships
This study proposes to capture and mimic navigator knowledge and
experiences by using deep learning, i.e. deep neural networks (DNNs), as a
ground-breaking technology that facilitates a better solution to control vessels.
A considerable number of sensors should also be on-board and the respective
data should be fused in a perception framework to achieve this objective.
In self-driving cars, the driver is successfully replaced by DNNs trained to
mimic human behavior, due to three main factors 1 :
collecting and analyzing large-scale real-world
driving data sets, including sensor and high definition
video/image data, to support deep learning based
digital drivers.
holistic understanding of how human drivers
interact with vehicle automation technologies by
observing video/image and vehicle motion data, driving
characteristics, human knowledge and experiences with
the new technologies during the training phase.
adequate safety buffer to save lives by identifying how
technology and other related factors can be used during
the self-driving phase, i.e. the execution phase.
1. A. Fridman, et al., MIT Autonomous Vehicle Technology Study : Large-Scale Deep Learning Based Analysis of
Driver Behavior and Interaction with Automation lessArXiv2017
UiT The Arctic University of Norway 7 / 25

8. Introduction Ship Intelligence Framework Ship Collision Avoidance Conclusions
Digital Helmsman
Human-AI-Technology-Regulations Interactions
UiT The Arctic University of Norway 8 / 25

9. Introduction Ship Intelligence Framework Ship Collision Avoidance Conclusions
Digital Helmsman
Deep Neural Networks (DNNs)
This is a considerable deviation from conventional control approaches in ship
navigation developed in the last decade.
Instead of control flow logics and if-then-else statements, DNNs consist of
state-of-the-art Neural Networks with many layers and millions to one billion
parameters, i.e. the weights of the respective neurons, of nonlinear activation
functions.
Convolutional neural networks are the most popular DNNs for self-driving cars
and those network parameters are adjusted via back-propagation type
approaches.
DNNs require a large amount of real-world vessel navigation data and
hundreds of thousands to millions of forward and backward training
iterations to achieve higher accuracy in navigator behaviour.
DNNs consist of large numbers of identical neurons with a highly parallel
structure that can be mapped to GPUs (Graphics processing unit) naturally to
obtain a higher computational speed when compared to CPUs based training.
UiT The Arctic University of Norway 9 / 25

10. Introduction Ship Intelligence Framework Ship Collision Avoidance Conclusions
Ship Intelligence Framework
SIF
UiT The Arctic University of Norway 10 / 25

11. Introduction Ship Intelligence Framework Ship Collision Avoidance Conclusions
Ship Intelligence Framework
SIF in The Training Phase
UiT The Arctic University of Norway 11 / 25

12. Introduction Ship Intelligence Framework Ship Collision Avoidance Conclusions
Ship Intelligence Framework
SIF in The Execution Phase
UiT The Arctic University of Norway 12 / 25

13. Introduction Ship Intelligence Framework Ship Collision Avoidance Conclusions
Collision Avoidance Features
Collision Avoidance Decisions
The respective Collision Risk in each ship encounter situation should be
estimated.
The same collision risk should be used to formulate appropriate Collision
Avoidance Decisions.
The respective collision avoidance decisions by humans and systems should have
a Good Consistency, since the digital helmsman is trained by human navigators.
The combination of those two components, i.e. the collision risk estimation and
collision avoidance decisions, can be categorized a decision support feature
and can be used by both humans and systems to make appropriate collision
avoidance actions.
Collision avoidance actions : Speed Change(to increase or decrease) and
Course Change (to the starboard and port).
UiT The Arctic University of Norway 13 / 25

14. Introduction Ship Intelligence Framework Ship Collision Avoidance Conclusions
Collision Avoidance Features
Collision Avoidance Actions
Collision avoidance actions should be executed through ship propulsion and
rudder control systems.
The alteration of course is the most preferred collision avoidance action, since
low speed conditions can also reduce ship maneuverability.
Since both humans and systems convert the collision avoidance decisions into
actions, their Regulatory Compliance should also be evaluated.
Any inconsistency between human and system collision avoidance actions
can be eliminated to preserve the integrity of system intelligence.
The respective regulatory compliance in relation to possible COLREGs failure
situations under several simulated system decision making situations in ship
navigation should be investigated.
That outcome can also help to improve digital helmsman behavior and identify
possible regulatory modifications to support future vessels.
UiT The Arctic University of Norway 14 / 25

15. Introduction Ship Intelligence Framework Ship Collision Avoidance Conclusions
Collision Risk Estimation
The Definition of Ship Collision
COLREGS, 1972 Rule 7 Risk of collision
a Every vessel shall use all available means
appropriate to the prevailing circumstances
and conditions to determine if risk of
collision exists. If there is any doubt such
risk shall be deemed to exist.
b Proper use shall be made of radar
equipment if fitted and operational,
including long-range scanning to obtain
early warning of risk of collision and radar
plotting or equivalent systematic
observation of detected objects.
c Assumptions shall not be made on the
basis of scanty information, especially
scanty radar information.
d In determining if risk of collision exists the
following considerations shall be among
those taken into account :
i such risk shall be deemed to exist if the
compass bearing of an approaching vessel
does not appreciably change;
ii such risk may sometimes exist even when
an appreciable bearing change is evident,
particularly when approaching a very large
vessel or a tow or when approaching a
vessel at close range.
UiT The Arctic University of Norway 15 / 25

16. Introduction Ship Intelligence Framework Ship Collision Avoidance Conclusions
Collision Risk Estimation
Decision Support Feature
Speed Change to increase (δVo > 0) or decrease
(δVo < 0) and Course Change to starboard (δψo > 0)
and port (δψo < 0).
UiT The Arctic University of Norway 16 / 25

17. Introduction Ship Intelligence Framework Ship Collision Avoidance Conclusions
Collision Risk Estimation
Safe Speed with Visibility & Lost Maneuverability
COLREGS, 1972 - Rule 6 Safe speed
Every vessel shall at all times proceed at a safe speed so that she can take proper
and effective action to avoid collision and be stopped within a distance appropriate
to the prevailing circumstances and conditions. In determining a safe speed the
following factors shall be among those taken into account :
a By all vessels :
i the state of visibility;
ii the traffic density including concentrations of fishing vessels or any other vessels;
iii the manoeuvrability of the vessel with special reference to stopping distance and turning ability in
the prevailing conditions;
iv at night the presence of background light such as from shore lights or from back scatter of her
own lights;
v the state of wind, sea and current, and the proximity of navigational hazards;
vi the draught in relation to the available depth of water.
b Additionally, by vessels with operational radar :
i the characteristics, efficiency and limitations of the radar equipment;
ii any constraints imposed by the radar range scale in use;
iii the effect on radar detection of the sea state, weather and other sources of interference;
iv the possibility that small vessels, ice and other floating objects may not be detected by radar at
an adequate range;
v the number, location and movement of vessels detected by radar;
vi the more exact assessment of the visibility that may be possible when radar is used to determine
the range of vessels or other objects in the vicinity.
UiT The Arctic University of Norway 17 / 25

18. Introduction Ship Intelligence Framework Ship Collision Avoidance Conclusions
Collision Avoidance
Collision Avoidance Actions at Any Cost
COLREGS, 1972 Rule 8 Action to avoid collision
a Any action to avoid collision shall be taken in accordance with the Rules of this Part and
shall, if the circumstances of the case admit, be positive, made in ample time and with
due regard to the observance of good seamanship.
b Any alteration of course and/or speed to avoid collision shall, if the circumstances of the
case admit, be large enough to be readily apparent to another vessel observing visually
or by radar; a succession of small alterations of course and/or speed should be avoided.
c If there is sufficient sea-room, alteration of course alone may be the most effective action
to avoid a close-quarters situation provided that it is made in good time, is substantial
and does not result in another close-quarters situation.
d Action taken to avoid collision with another vessel shall be such as to result in passing at
a safe distance. The effectiveness of the action shall be carefully checked until the other
vessel is finally past and clear.
e If necessary to avoid collision or allow more time to assess the situation, a vessel shall
slacken her speed or take all way off by stopping or reversing her means of propulsion.
i A vessel which, by any of these Rules, is required not to impede the passage or safe passage of
another vessel shall, when required by the circumstances of the case, take early action to allow
sufficient sea-room for the safe passage of the other vessel.
ii A vessel required not to impede the passage or safe passage of another vessel is not relieved of
this obligation if approaching the other vessel so as to involve risk of collision and shall, when
taking action, have full regard to the action which may be required by the Rules of this part.
iii A vessel the passage of which is not to be impeded remains fully obliged to comply with the
Rules of this part when the two vessels are approaching one another so as to involve risk of
collision.
UiT The Arctic University of Norway 18 / 25

19. Introduction Ship Intelligence Framework Ship Collision Avoidance Conclusions
COLREGs
Head-on Situation
COLREGS, 1972 Rule 14 Head-on situation :
a When two power-driven vessels are meeting on
reciprocal or nearly reciprocal courses so as to
involve risk of collision each shall alter her
course to starboard so that each shall pass on
the port side of the other.
b Such a situation shall be deemed to exist when
a vessel sees the other ahead or nearly ahead
and by night she could see the masthead lights
of the other in a line or nearly in a line and/or
both sidelights and by day she observes the
corresponding aspect of the other vessel.
c When a vessel is in any doubt as to whether
such a situation exists she shall assume that it
does exist and act accordingly.
UiT The Arctic University of Norway 19 / 25

20. Introduction Ship Intelligence Framework Ship Collision Avoidance Conclusions
COLREGs
Overtaking Situation
COLREGS, 1972 Rule 13 Overtaking
a Notwithstanding anything contained in the Rules
of part B, sections I and II, any vessel overtaking
any other shall keep out of the way of the vessel
being overtaken.
b A vessel shall be deemed to be overtaking when
coming up with another vessel from a direction
more than 22.5 degrees abaft her beam, that is,
in such a position with reference to the vessel
she is overtaking, that at night she would be
able to see only the sternlight of that vessel but
neither of her sidelights.
c When a vessel is in any doubt as to whether she
is overtaking another, she shall assume that this
is the case and act accordingly.
d Any subsequent alteration of the bearing
between the two vessels shall not make the
overtaking vessel a crossing vessel within the
meaning of these Rules or relieve her of the duty
of keeping clear of the overtaken vessel until she
is finally past and clear.
UiT The Arctic University of Norway 20 / 25

21. Introduction Ship Intelligence Framework Ship Collision Avoidance Conclusions
COLREGs
Crossing Situation
COLREGS, 1972 Rule 15 Crossing
situation
When two power-driven vessels are
crossing so as to involve risk of
collision, the vessel which has the
other on her own starboard side shall
keep out of the way and shall, if the
circumstances of the case admit, avoid
crossing ahead of the other vessel.
COLREGS, 1972 Rule 16 Action by
give-way vessel
Every vessel which is directed to keep
out of the way of another vessel shall,
so far as possible, take early and
substantial action to keep well clear.
UiT The Arctic University of Norway 21 / 25

22. Introduction Ship Intelligence Framework Ship Collision Avoidance Conclusions
COLREGs
Crossing Situation (Cont.)
COLREGS, 1972 Rule 17 Action by stand-on
vessel
a i Where one of two vessels is to keep out of the
way the other shall keep her course and speed.
ii The latter vessel may however take action to avoid
collision by her manoeuvre alone, as soon as it
becomes apparent to her that the vessel required
to keep out of the way is not taking appropriate
action in compliance with these Rules.
b When, from any cause, the vessel required to
keep her course and speed finds herself so
close that collision cannot be avoided by the
action of the give-way vessel alone, she shall
take such action as will best aid to avoid
collision.
c A power-driven vessel which takes action in a
crossing situation in accordance with
subparagraph (a)(ii) of this Rule to avoid
collision with another power-driven vessel shall,
if the circumstances of the case admit, not alter
course to port for a vessel on her own port side.
d This Rule does not relieve the give-way vessel of
her obligation to keep out of the way.
UiT The Arctic University of Norway 22 / 25

23. Introduction Ship Intelligence Framework Ship Collision Avoidance Conclusions
COLREGs
Possible Regulatory Failures
UiT The Arctic University of Norway 23 / 25

24. Introduction Ship Intelligence Framework Ship Collision Avoidance Conclusions
The End
Conclusions
It is extremely difficult to create a ship collision situation.
A slight change in course and/speed conditions can avoid
a possible collision situation.
An intelligent collision avoidance framework is presented
and possible regulatory failures.
Regulatory failures can occur as intermediate states of
ship encounter situations and cannot be ignored.
In realistic ocean-going situations, the following outcomes
are expected :
If the course-speed and heading vectors in these vessels are
having significant differences; therefore these vessels can be
moving in parabolic type maneuvers; then that can introduce
additional complexities into the collision risk estimation.
If this situation is associated with more than two vessels,
various collision avoidance decisions can overlap and even
cancel each other.
If any constrains in the vessel maneuverability can introduce
additional difficulties in executing collision avoidance actions.
That can degrade the collision avoidance
decisions/actions that are taken by humans and systems.
System intelligence and regulatory modifications are yet
to be investigated.
UiT The Arctic University of Norway 24 / 25

25. Introduction Ship Intelligence Framework Ship Collision Avoidance Conclusions
The End
Any Question?
FIGURE – IST Autonomous Vessel, Lisbon, Portugal
URL : https ://www.youtube.com/user/thecentec
Publications : http ://bit.do/maritime
FIGURE – UiT Bridge Simulator & Autonomous Vessel, Tromso,
Norway
UiT The Arctic University of Norway 25 / 25