745

1 INTRODUCTION

General collision avoidance rules between two or

more vessels are described in convention known as

the COLREGs‐the International Regulations for

Preventing Collisions at Sea, published by IMO

(International Maritime Organization) at London on

20 October 1972. It replaced similar previous

regulationsof1960.TheConventioncontainsrules

of

the road for sea‐going vessels, definitions of ships,

arrangement of lights and shapes, description of

soundandlightsignals.

It is important to mention that regarding

COLREGs there is no absoluteʺright of wayʺ

privilege of a sea‐going vessel over any other

encountered vessel. It is rather

described asʺgive

wayʺ(burdened)vesselandaʺstandonʺ(privileged)

vessel, sometimes there may be two “give way”

vesselswithno“standon”vessel.Astandonvessel

maystillbeobligedto giveway,ifthereisa risk of

collision. A commander on the bridge shall never

assume

theothervesselshareshisviewofwhichrules

apply in particular situation. The decision is not

always clear, it may differ depending on weather

condition, visibility, experience and many other

factors. It may be aided by intelligent decision

supportsystemwithfuzzyvalues(Pietrzykowskiand

Małujda, 2012). There

are also solutions for

unmanned vessels (Naeem et al., 2012) and finding

safeshiptrajectorieswithfocusonbetterhandlingof

COLREGS (Szłapczyński and Sz łapczyńska, 2012).

Interestingapproachtomulti‐shipcollisionavoidance

underCOLREGsregulationshasbeenproposedwith

usage of Artificial Potential Field method (Wang

 et

al.,2017)tocalculateshipssafetydomain.

Neuroevolutionary Approach to COLREGs Ship

Maneuvers

M.Łącki

GdyniaMaritimeUniversity,Gdynia,Poland

ABSTRACT:Thepaperdescribestheusageofneuroevolutionarymethodincollisionavoidanceoftwopower‐

drivenvesselsapproachingeachotherregardingCOLREGsrules.Thismaybealsobeseenastheshiphandling

systemthatsimulatesalearningprocessofagroupofartificialhelmsmen‐autonomous

controlunits,created

withartificialneuralnetworks.Thehelmsmanobservesanenvironmentbyitsinputsignalsandaccordingto

assignedCORLEGs rule, hecalculates the values of requiredparameters of maneuvers (propellers rpm and

rudderdeflection)inacollisionavoidancesituation.Inneuroevolutionsuchunits aretreatedasindividualsin

population of artificial neural networks, which through environmental sensing and evolutionary algorithms

learntoperformgiven task safelyand efficiently. The main task of this projectis to evolvea population of

helmsmenwhichisabletoeffectivelyimplementchosenrule:crossingorovertaking.



http://www.transnav.eu

the International Journal

on Marine Navigation

and Safety of Sea Transportation

Volume 13

Number 4

December 2019

DOI:10.12716/1001.13.04.06

746

2 NEUROEVOLUTION

Neuroevolution is an artificial intelligence method

that uses evolutionary algorithms (EA) to generate

artificial neural networks (ANN), its parameters,

topology and rules. Such combination gives the

advantage of flexibility and adaptability, which

allows to adjust the computational structures to the

dynamicallychangingconditionsencounteredduring

ship maneuvering and

 are intensively studied and

implementedin many differentfields of science and

research, including robotics (Lee et al., 2013),

automation processes (Stanley et al., 2005), multi‐

agent systems (Nowak et al., 2008) designing and

diagnostics(Larkinetal.,2006),andmanymore.

Neuroevolutionary algorithms are successful

methods for optimizing neural

networks topologies

fordynamiccontinuousreinforcementlearningtasks.

Their significant advantage over gradient‐based

algorithms is the capability to modify network

topologies along with connection weights, resulting

withbroadersearchspaceofpossiblesolutions.

The proper maneuvers of ship maneuvering

accordingtoCOLREGsrulesisessentialtothesafety

of people,

 equipment, cargo and the environment.

Increaseofcomputationalpowerofelectronicdevices

allows to implement as complex algorithms as

neuroevolution into advanced decision support

systems(DSS)alsointhefieldofmarinenavigation.

Throughcontinuousenvironmentobservationand

learning process, such DSS shall predict the vessels’

rudderangleandpropeller

revolutionsasaccurate as

possibletoensuresafeimplementationofchosenrule.

It is possible to calculate these output signals when

thereisasimulationmodelofthevesselavailable.

InneuroevolutionANNistreatedasanindividual

inapopulationofmultiplenetworks.Neuroevolution

is able to find a solution

 of a  complex and

dynamically changing task with ANN created and

modifiedwithEA. Thebasictopologiesof theinitial

populationarerandomlydeterminedatthebeginning

oflearningprocess.Eachindividualactsasabrainof

artificialhelmsmanandbeginstheprocessoffinding

a solution with the same

starting parameters (ships

initial position, course, velocity, rudder angle, rpm).

Theactionofeachindividualisusuallyassessedwith

thereinforcementlearningalgorithms(Stanleyetal.,

2005)andevolutionarystageofthesystemshallselect

individuals best suited to the task during selection

stage, which determines the whole population to

improve

itsgeneticmaterialovertime.

Evaluation of each individual is being processed

duringwholesimulationaftersomeimportantevents

tookplace,asforexample:

 movingthevesseloutoftheareaoronforbidden

sector, i.e. the safety domain of an encountered

vessel,

 making rapid and/or frequently changing

maneuvers,i.e.tofrequentalterationofrpm,

 leadingtoimproper ships’ movement parameters

values,i.e.linearand/orangularvelocity too low

ortoohigh,

 movingthevesselawayfromgoal,

 reachingagoal.

Alltheseeventmustbearbitrarilyrated,resulting

inarewardtoevaluatedindividual,thus

valuatingits

fitness important in evolutionary stage of the

algorithm and consequently his chance of

reproductionandsurvivaltothenextgeneration.

Evolutionary process of the system consists three

mainsteps:

 selectionofthebestindividualorindividuals,

 reproduction (with cross‐over and mutation sub‐

processes),

 replacement

(offspring replaces worst

individuals).

Forthepurposeofthistasktheneuroevolutionary

method, the modified NEAT algorithm, with direct

encoding of neural network topology has been

implemented.

NEAT (NeuroEvolution of Augmenting

Topologies) (Stanley and Risto, 2002) adjust the

topology of ANN’s with EA gradually, allows to

obtainasetofindividuals

thatarebestfittedtogiven

task.

Input and output signals of ANN’s have been

determinedatthebeginningofdesigningphaseofthe

system.Properlydesignedsetofsignalsconsideredin

the model is crucial for efficiency of the system as

muchasforitsfidelityandaccuracyin

comparisonto

therealnavigationalsituation.

Inputsignalsinthesystem,withthreedegreesof

freedomofthevesselmovement,areasfollows:

 ships’courseoverground,

 ships’angularvelocity,

 ships’speedoverground,

 ships’position,

 ships’distanceandangletogoal,obstaclesandto

the

encounteredvessel,

 mainpropellerrevolutions(currentandpreset),

 rudders’deflection(currentandpreset).

Infutureresearchothersignalsfromenvironment

maybetakenintoaccount,i.e.wind,current,waves,

cargo,trimandroll,ifdeliveredinashipmodel.

Outputsignals of ANNs generates the values for

steering

thevessel:

 rpmofmainpropeller,

 rudders’deflection.

Alloftheinputandoutputsignalsarenormalized

andencodedasrealvaluesbetween0and1.

Each node in ANN represents a neuron that

producesarealvaluebetween0and1asaresultof

normalized weighted sum

 of its inputs.

Normalization of weighted sum is performed with

sigmoidfunction.

The simulation results are shown below for

simulationmodel of three‐degrees‐of‐freedom VLCC

crude oil tanker “Esso Norway” with the single‐

propellerandsingle‐rudder(Figure1).

747



Figure1.VLCCoiltanker„EssoNorway” (1969‐1986)

Mainparametersofthesimulatedvesselhasbeen

placedintable1.

Table1.Mainparametersof“EssoNorway”

______________________________________________

ParameterValue

______________________________________________

Lengthoverall323,8m

Lengthbetweenperpendiculars  304,8m

Beam47,3m

Max.draft18,46m

Deadweighttonnage193048t

Max.revolutionsofpropeller80rpm

Max.rudderdeflection±20°

_____________________________________________



Inthissimulationithasbeenassumedthat“Esso

Norway”willencountersecondvesselofsimilarsize

and heading forward on steady course. Her safety

domain has been established as simplified rectangle

shape 3 length ahead of her bow and one length

behindthestern(Figure2).Widthofthisdomain

is2

lengthofthevessel.



Figure2.Simplifiedsafetydomainoftheencounteredvessel

3 SIMULATIONOFCOLREGSMANEUVERS

3.1 Rule13–Overtaking

Rule13statesthat:

“a)Notwithstandinganything containedintheRules

ofPartB,SectionsIandII,anyvesselovertakingany

other shall keep out of the way of the vessel being

overtaken.

(b)A vessel shall be deemed to

be overtaking when

coming up with a another vessel from a direction

morethan22.5degreesabaftherbeam,thatis,insuch

a position with reference to the vessel she is

overtaking,thatatnightshewouldbeabletoseeonly

the sternlight of that vessel but neither

of her

sidelights.

(c)Whenavesselisinanydoubtastowhethersheis

overtaking another, she shall assume  that this is the

caseandactaccordingly.

(d)Anysubsequentalterationofthebearingbetween

thetwovesselsshallnotmaketheovertakingvessela

crossingvesselwithinthe

meaningoftheseRulesor

relieve her of the duty of keeping clear of the

overtakenvesseluntilsheisfinallypastandclear.”

Overtakingsituationispresentedonfigure3.



Figure3.Initialsituationofovertakingmaneuvers

VesselAisheadingnorthfaster(5m/s)thanvessel

B (3 m/s), thus overtaking maneuver occurs. In this

case it is strongly recommended that vessel A shall

overtakeBonherportside.

748



Figure4. An example of recorded routes of a whole

population

As one can see on figure 4 there is a broad

spectrumofroutestakenontheportsideofovertaken

vessel B. Many of them end width failure upon

enteringothervessels’safetydomain,ormovingout

of the area. Some routes end within the goal with

improper course over

 ground, but fitness values of

these individuals are better than those leaving the

area and their chance of survival to the next

generation is greatly higher. It is of course not

sufficient to reproduce as intense as the best

individuals but is not totally excluded and has a

chanceto

improveitsfitness  insubsequent episodes

ofsimulationprocess.



Figure5.Finalrouteofthebestindividual

The results of simulation is the route of the best

individual, chosen after specified duration of

simulation. His actions taken and maneuvering

parametersvalueshasbeenpresentedinfigures6‐10.



Figure6.Suggestedandactualrudderangle

The system calculated 3 rudder angle changes,

resultingchangesinangularvelocityaswell.



Figure7.Angularvelocity



Figure8.Linearvelocityinm/sandinknots



Figure9.Suggestedandactualrevolutions[rpm]

Propeller revolutions changes suggested by the

system seem to be too low, causing in linear speed

decrease below 4 m/s [7 kn]. It may be solved in

further simulation by the adjustment of a reward

valuerelatedtoships’velocityortimeoftravel.



Figure10.Courseoverground

3.2 Rule15–CrossingSituation

Crossingmaneuversaredescribedinrule15:

“Whentwopower‐drivenvesselsarecrossingsoasto

involve risk of collision, the vessel which has the

otheronherownstarboardsideshallkeepoutofthe

wayandshall,ifthecircumstancesofthe

caseadmit,

avoidcrossingaheadoftheothervessel.”

An example of simulation process of crossing

situation is presented on figure 11. Give‐way  vessel

749

starts with course 110° on the port side of the

encounteredvesselwhichisheadingnorth.



Figure11. An example of simultaneous positions of each

individualinpopulationofhelmsmenduringsimulation

Duringsimulationeachartificialhelmsmantriesto

safelynavigatethroughareatoagoal,andaccording

torule15,hetriestoavoidcrossingaheadoftheother

vesselduringmaneuvers.



Figure12. An example of recorded routes of a whole

population

There are some individuals that prefer relatively

safe circulation than risky maneuvering near

forbidden domain of the encountered vessel. Their

fitnessisslightlybetterthanonesleavingthearea,but

they are still too weak to compete with better units

duringselection.



Figure13.Finalrouteofthebestindividual

The results of simulation are presented as final

route of the best individual and its actions and

parametersinfigures14‐18.



Figure14.Suggestedandactualrudderangle



Figure15.Angularvelocity



Figure16.Linearvelocityinm/sandinknots

750



Figure17.Suggestedandactualrevolutions[rpm]



Figure18.Courseoverground

Shipofthatsizehas verylimitedmaneuverability,

what implies higher awareness of early orders  of

changing her rudder deflection or/and propeller

thrust.Simulationresultsshowthatchangeofthrust

has no big impact of ships’ speed in considered

situations.Moreimportantis rudder angle, which is

limitedto±20°.

REMARKS

ManydiscussionsonCOLREGshaveraisedandbeen

continued since its creation and first submission

relatedtoitsapplication.Therearesomeproposalsof

improvements in the maritime education to reduce

the negative impacts during the implementation of

theCOLREGs(DemirelandBayer,2015).

Neuroevolutionary approach to ship handling

duringcross‐overorovertakingofencounteredvessel

may improve a quality of maneuvers and safety of

navigation. For the simulation study, mathematical

model of three‐degrees‐of‐freedom VLCC crude oil

tanker with the single‐propeller and single‐rudder

hasbeenusedtotesttheperformanceofthesystem.

 comparison to a classic state machine learning

algorithms(Łącki,2007)theartificialneuralnetworks

based on modified NEAT method may increase

complexity and performance  of considered modelof

ship maneuvering according to COLREGs.

Neuroevolutionaryshiphandlingsystembringssome

valuablebenefitstothisapproach:

 increaseofthesafety

ofnavigationby improving

the data analysis for decision‐maker during

maneuvers,

 reduction of operating costs of vessels due to

reduced number of extreme and unnecessary

maneuvers,

 minimizationoftheoccurrenceofhumanerrors,

 reduction of the harmful impact of transport on

theenvironment

 finding some new

 solutions related to heuristic

characteristicsofneuroevolution.

It is important to notice that all these benefits in

neuroevolutionstrictlydependonproperadjustment

ofevolutionaryparametersandprocesses,thesizeof

ANNs population and the encoding methods of

signalsconsideredinservicedenvironment.

Successfulsimulationresultsencouragetofurther

research

of the neuroevolutionary methods with

additional disturbances from the influence of sea 

waves, ocean currents and winds, for different ship

models,whichmaybesuccessfullyimplementedinto

advancednavigationalsystemstoincreasethesafety

ofnavigation.

REFERENCES

Demirel,E. &Bayer, D. 2015. The Further Studies OnThe

COLREGs (Collision Regulations), TransNav, the

InternationalJournalonMarineNavigationandSafetyofSea

Transportation,vol.9,17–22.

Łącki, M. 2007. Machine Learning Algorithms in Decision

Making Support in Ship Handling. , Katowice‐Ustroń:

WKŁ,p.

Larkin, D., Kinane, A. &  O’Connor, N. 2006. Towards

hardware acceleration of neuroevolution for multimedia

processing applications on mobile devices , Hong Kong,

China.

Lee, S., Yosinski, J., Glette, K., Lipson,  H. & Clune J 2013.

EvolvinggaitsforphysicalrobotswiththeHyperNEAT

generative encoding: the benefits of simulation,

Applications

ofEvolutionaryComputing,.

Naeem, W., Irwin, G.W. & Aolei, Y. 2012. COLREGs‐based

collisionavoidancestrategiesforunmannedsurfacevehicles,

Oxford,ROYAUME‐UNI:Elsevier.

Nowak, A., Praczyk, T. & Szymak, P. 2008. Multi‐agent

system of autonomous underwater vehicles‐

preliminary report, Zeszty Naukowe Akademii Marynarki

Wojennej,vol.4,

99–108.

Pietrzykowski,Z.&Małujda,R.2012.Applicabilityoffuzzy

logic to the COLREG rules interpretation, Zeszyty

Naukowe/AkademiaMorskaWSzczecinie,109–114.

Stanley, K.O. & Risto, M. 2002. Efficient Reinforcement

LearningThroughEvolvingNeuralNetworkTopologies.

Stanley, K.O., Bryant, B.D. & Risto, M. 2005. Real‐time

neuroevolution in the

 NERO video game, IEEE

TransactionsonEvolutionaryComputation,vol.9,653–668.

Szłapczyński,R.&Szłapczyńska,J.2012. EvolutionarySets

of Safe Ship Trajectories: Evaluation of Individuals,

TransNav, the International Journal on Marine Navigation

andSafetyofSeaTransportation,vol.6,345–353.

Wang,T.,Yan,X.P.,

Wang,Y.& Wu,Q.2017.ShipDomain

Model for Multi‐ship Collision Avoidance Decision‐

making with COLREGs Based on Artificial Potential 

Field, TransNav, the International Journal on Marine

NavigationandSafetyofSeaTransportation,vol.11,85–92.

