375
1 INTRODUCTION
The development of autonomous ships is
accompanied by an increased focus on enhanced
decision support systems (DSS). These systems
support the safe operation of ships by performing
various tasks onboard to reduce the workload of
navigators[1].Increasingcomplexityofsuchsystems,
includinginteractionsbetweenhumans,software,and
hardware, may lead to emergent behavior that is
difficulttopredict,detect,andmitigate.
Verificationreferstothe process of evaluating or
providingevidenceofasystem’sabilitytosatisfyits
requirements [2]. In the maritime industry,
classification societies set guidelines and procedures
for verification. For ship control and monitoring
systems onboard traditionally manned ships,
verification is a three stage process [3], consisting of
software verification, certificationtesting of software
and hardware, and onboard testing focused on
equipment functionality and communication. For
verificationofautonomousnavigationsystems(ANS),
DNV proposes additional yet complementary
guidelines[4].Verificationtestingmustbeperformed
in
twoareas:1)redundancyandfailureresponsetests,
and 2) testing of integrated systems and functions.
Redundancy and failure response tests are typically
generated from the results of a failure mode and
effects analysis (FMEA), while the methods to
generate integration tests and the procedures to
evaluatetheirresultsare
ambiguous.
Current methods for testing of complex systems
aretypicallyderivedfromhardware‐in‐the‐loop(HIL)
testing. HIL allows for systematic testing of system
behaviorthroughtheaidoforuseofsimulation[5].In
themaritimeindustry,HILtestinghaspredominately
Integration Test Procedures for a Collision Avoidance
Decision Support System Using STPA
S.A.Dugan
1
,R.Skjetne
1
,K.Wróbel
2
,J.Montewka
3
,M.Gil
2
&I.B.Utne
1
1
NorwegianUniversityofScienceandTechnology,Trondheim,Norway
2
GdyniaMaritimeUniversity,Gdynia,Poland
3
GdańskUniversityofTechnology,Gdańsk,Poland
ABSTRACT: The transition from conventionally manned to autonomous ships is accompanied by the
development of enhanced Decision Support Systems (DSS) for navigators. Such systems need to consider
interactionsamonghardware,software,andhumansandtheirpotentialeffectsonsystemperformance,which
require
rigorous testing to verify the systemʹs safe decision‐making ability and operational limits. Testing
requirementsforverificationareaimedat1)assessingthesystemʹsreliabilityandfailurehandlingperformance,
and2)integrationtesting.ThisworkusestheSystem‐TheoreticProcessAnalysis(STPA)todevelopintegration
testsforanovel
DSS.STPAisastructuredmethodologytoidentifyhazardsfrommultiplesources,including
hardwareorsoftwarefailures,systeminteractions,andhumanerrors.Theobjectivesofthestudyaretodevelop
and assess the feasibility of integrationtest procedures based on STPA. The stability monitoring subsystem
from the DSS is analyzed
as a case study. The results are used to suggest functional and performance
integrationtestprocedures.
http://www.transnav.eu
the International Journal
on Marine Navigation
and Safety of Sea Transportation
Volume 17
Number 2
June 2023
DOI:10.12716/1001.17.02.14
376
been used for dynamic positioning (DP) system
testing [5, 6]. For verification, an HIL test scope
consistsofseveraltypesoftestingincludefunctional,
failuremode,andperformancetesting[6].Functional
testingassessesthecomplianceofasystem’sfunction
to its functional requirements. Performance testing
quantifiesthelevelofperformance
ofafunction[5].
Simulation has been identified as a method of
enablinglargescale testsofscenariosforverification
[7]. Simulation‐based testing extends from the
principles of HIL testing, and typically relies on
comprehensivemodels of theship,theenvironment,
and a test management system [7]. The test
management system is responsible for generation of
test scenarios, specifying acceptance criteria, and
evaluation of the results. Although simulation‐based
testingallowsfortestingsystembehaviorinmultiple
scenarios, questions exist on its performance.
Specifically,howshouldanefficientandrelevanttest
scopebedetermined,andwhatproceduresshouldbe
used
toevaluatethetestresults[8].
TheSystem‐TheoreticProcessAnalysis(STPA)has
beenincreasinglyfeaturedinresearchonverification
andvalidationofmaritimeautonomoussurfaceships
(MASS). STPA is a hazard assessment methodology
basedoncontroltheory[9]andhasbeenidentifiedas
particularlysuitableforevaluatingthesafety
levelof
complex systems [10]. The method focuses on the
controlstructureofasystemtobetterunderstandthe
systembehavior.Comparedtootherhazardanalysis
methods,STPAisnotedforitsabilitytoidentifythe
interactionsbetweendifferentfailuretypes[11,12].
The “systems‐level” perspective of the STPA has
alsoresultedinidentifyingseveralaccidentscenarios
not captured by FMEA, which often investigates
safety at a component level [13]. Rokseth et al. [14]
uses STPA to analyze the safety of DP systems by
deriving verification objectives based on the
requirementselucidatedbySTPA.Roksethetal.[15]
expands the
method to develop a verification
program at different stages of system development,
including suggesting test procedures for system
performance. For each loss scenario, the aim, setup,
execution, and acceptance criteria are provided for
multiplestagesinthesystem’sverificationprocess.
The research nevertheless indicates a gap in
presentingguidelinesforidentifying
and conducting
testsforsystemintegration.Therefore,theobjectiveof
thispaperisto applyamethodologyfordeveloping
such tests based on the results of an STPA. The
principles of HIL testing are extended to improve
verification objectives and procedures. The objective
oftheworkistopresentthe
developmentoftargeted
functionalandperformancetestsforaDSSdeveloped
as part of the Endure research project (project‐
endure.eu). The purpose of the tests is to identify
operational limits and practical settings for the
system’s operation. The approach is similar to that
followed in Rokseth et al. [15]. However, instead
of
presenting verification objectives, we focus on
functional and performance based aspects of
integrationtesting.
Thepaperisorganizedasfollows:thenextsection
presents the methodology of the study, including
STPA and the proposed approach to develop
integration tests. The system of study is then
described.Resultsincludethe
stepsoftheSTPAand
the derivation of example integration tests for a
selected loss scenario. The discussion focuses on the
impactoftheresults forsystem verification, andthe
feasibilityofSTPAforgeneratingtestprocedures.
2 METHODOLOGY
2.1 Step1:STPA
STPA is a hazard analysis method derived from
the
System Theoretic Accident Model and Processes
(STAMP) [16]. Before describing the purpose and
methodologyofSTPA,afewwordsarededicatedto
itsprecursor.
STAMP is an accident causality model that shifts
the emphasis from preventing failures to enforcing
behavioral safety constraints [16]. Although
component failure accidents are still
included, the
ability to analyze and understand component
interactions leads to a better understanding of the
morecomplex systems oftoday. The introductionof
the STAMP model requires the refinement of
terminology, which is consistent in its related
techniques.Anaccidentistermedanunplannedand
undesired loss event [16]. Safety
is re‐framed to a
problem of control: safety constraints are placed to
restrict emergent properties between component
interactions.STAMPanditsrelatedtechniques,STPA
andCausalAnalysisbasedonSTAMP(CAST),have
been used to analyzeaccidents andsystemsranging
fromaircraftcontroltoaquaculture[10].
STPAischosen
asthemethodforthisanalysisin
ordertocapturethecomplexfunctionalrelationships
between controllers, especially when emergent
behaviorsandcompetingobjectivesareexpectedtobe
revealed. Additionally, it can be used to model
multiple types of controllers (human, software,
hardware)andtheinteractionsbetweenthem.
Thefourstepsof
STPAaredetailedbelow[9]:
1. Define the purpose of the analysis. This step
contains four parts: identify losses, identify
system‐level hazards, identify system‐level
constraints, and refining hazards (optionally). In
thisstage,itisimportanttodescribetheboundary
ofthesystemwithinitsenvironment.
2. Model the
hierarchical control structure. The
control structure describes the functional
relationships and interactions within the system
throughtheuseoffeedbackcontrolloops.
3. Identify unsafe control actions (UCAs), or control
actionsthatinparticularenvironmentsandcontext
will leadto a hazard [9]. UCAs are then used to
create the requirements
and constraints for the
system.Thereare fourways that a control action
canbeconsideredunsafe[9]:
Not providing the control action leads to a
hazard
Providingthecontrolactionleadstoahazard
Providing a potentially safe control action too
early,toolate,orin
thewrongorder
Thecontrolactionlastingtoolongorstopping
toosoon
4. Identify loss scenarios (LS). The purpose of the
scenarioidentificationistwofold:
377
to demonstrate how factors within the control
structurecauseUCAsandleadtolosses
how safe control actions might be executed
improperlyandleadtolosses
These losses can further contribute to the
developmentofrequirementsandconstraintsforthe
system.
2.2 Step2:Integrationtestdevelopment
The
purposeofintegrationtestingistoprovethatno
emergent properties between system functions and
their dependencies will degrade the system. Within
HILtesting, two types of testsare primarilyused to
accomplish this: functional testing and performance
testing [5, 6]. Functional testing is the testing of a
function’sability
tofulfil itsrequirements[5]. In the
context of integration, this requires investigating the
structureandprocessvariables(PVs)ofthesystemto
identify any dependencies that may contribute to a
function’sfailure.Typically,performancetestingaims
to quantify the performance level for a function.
Again,withinthecontextofintegration,
performance
testinginvestigatestheimpactofdifferentconditions
(both internal and external) on the function’s
performance[5].
Integration testing is focused on the interactions
between components in the system. We use the
hierarchicalcontrolstructurefromtheSTPAasabasis
forunderstandingthebehaviorofacontrollerwithin
the
context of the system. We identify interfaces by
modelingtheinputsandoutputsofthecontroller.The
unsafecontrolactionsderivedfromSTPAareusedto
generate safety constraints, or requirements, for
functional testing. Loss scenarios provide insight on
the dependencies or conditions that may lead to
failures. These conditions are
used to derive
performancetests.Integrationtestproceduresinclude
descriptionsofthesetup,execution,andevaluationof
theresultsinaccordancewithIEEErequirements[2].
Todevelopatestplanbasedontheresultsofthe
STPA,weperformthefollowingadditionalsteps:
1. Foranunsafecontrolaction(STPA,
Step3),
Identify PVs that contribute to the unsafe
controlaction.
Developfunctionaltestbasedonthegenerated
safetyrequirement(s).
2. Foralossscenario(STPA,Step4),
IdentifyPMVconditionsthatcontributetothe
LS.
Select key performance indicator(s) (KPIs) to
evaluatefunctionperformance.
Develop
performance tests for the identified
PMVconditions.
Examplesof functionalandperformance testsare
presentedinSection4.
3 SYSTEMOFSTUDY
The methodology is applied to a DSS for collision
avoidance.TheDSS isnovelduetoitsconsideration
of vessel intact stability when evaluating evasive
maneuversforcollision
avoidance.Thisismotivated
by the role ofintact stabilityin two recent maritime
accidents:thecapsizingsoftheGoldenRay[17]and
MV Sewol. The latter is one of the worst maritime
disasters in recent history, with a death toll of 306
passengers[18].
Intact stability is governed by the
interaction of
weight and buoyancy [19]. The two primary
parametersofinterestarethelocationsofthecenters
ofgravityandbuoyancy.Theweightandlocationof
theship’scenterofgravityisestimatedbytheofficer
ofthewatch(OOW)beforeeachvoyage.Thecenterof
buoyancyisa
functionof theship’sweightandhull
shape. This information is typically used by the
onboard stability computer to estimate the ship’s
intactstability.
Turning maneuvers, particularly at high speeds,
createlargeheelingmomentsonthevesselthatmay
cause capsizing [20]. Heeling moments act to move
the ship away from
the upright position. At large
anglesofheel,thefactorsinfluencingtheprobability
ofexperiencinglargerollmotionsincludeshipspeed,
rudderangle,andwaveheight,period,anddirection
[21].
TheDSSsystem,developedaspartoftheEndure
project, will be installed on the training vessel
HoryzontII.The
shipprofileviewisshowninFigure
1.Principalshipcharacteristicsarebelow:
Lengthoverall:56.34m
Lengthbetweenperpendiculars:48.37m
Breadth:11.36m
Blockcoefficient:0.60
Servicespeed:12knots
Figure1.HoryzontIIprofileview
Thesystemperformsconditiondetectionutilizing
leadingsafetyindicators[22],conditionanalysis,and
action planning, but has no direct control over the
ship’smotion.Forobjectdetection,thesystemextracts
ship speed, type, and size from static and dynamic
AISdata. The OOW canmanuallyinput objects that
are not detected
or detectable by AIS. Condition
analysis,thenextphase,isperformedbyassessingthe
presenttargetship’sbehaviorandplannedownship’s
actioninordertoselectsuitable,predefinedcollision
avoidance dynamic critical areas (CADCA) , pre‐
computed for own ship encounters with the target
ship’ssafetydomain.Objectclassification
isusedfor
the situational analysis of potential conflicts. Lastly,
action planning is performed by restricting the
CADCAs to only consider evasive maneuvers with
rudderanglesthatdonotjeopardizetheship’sintact
stabilityforgivenwaveconditions.
378
4 RESULTS
4.1 Step1.STPA
TheSTPAwasperformedbyresearcherswithdomain
knowledgeandinvolvedwiththeDSS’sdevelopment.
System architecture diagrams and flowcharts were
consulted to model the system control structure
diagramandbetterunderstandthesystem’sbehavior.
4.1.1 Step1.0:Definethesystemboundary
Thesystem
ofstudyisrestrictedtothebehaviorof
theownship(OS).ThiscomprisestheOOW,theDSS
with its sensors and components, and the ship
(including its propulsion and maneuvering
equipment). The environment therefore consists of
surroundingshipsandobstacles.Evasivemaneuvers
are restricted to turns; speed reduction is
not
consideredasapossiblemaneuver.
4.1.2 Step1.1:Definethepurposeoftheanalysis
ThepurposeoftheDSSistoprovideinformation
totheOOWregardingpotentialcollisionswithother
ships and to prevent excessive maneuvers that may
jeopardize the ship’s intact stability. Therefore, the
accidents of investigation are
collision with an
obstacle and stability failure. The issues of
cybersecurity and intentional attacks (i.e., arson or
vandalism)arepresentlyexcludedtoreducethescope
oftheanalysis.Thelosses(L)aretherefore:
L‐1:Theshipcollideswithanobstacle.
L‐2:Theshipcapsizes.
System level
hazards that may lead to the losses
arelistedbelow.Theparenthesis indicatethe lossto
whichthehazard(H)maylead.
H‐1:TheshipviolatestheCADCAfortheobstacle
(L‐1).
H‐2: The ship violates the minimum stability
requirement(L‐2).
The hazards were refined
to consider two causal
scenarios of each failure type. First, we consider
hazards in which faulty or invalid commands are
provided.TheseleadtoaviolationofCADCAorthe
minimum stability requirement. Next, we consider
hazards in which the correct commands are not
provided.
H‐1.1: Motion control
commands that result in
violation of the CADCA for an obstacle are
provided(L‐1).
H‐1.2: Motion control commands that result in
preservationoftheCADCAforanobstaclearenot
provided(L‐1).
H‐2.1: Motion control commands that result in
violationoftheminimumstabilityrequirement
are
provided(L‐2).
H‐2.2: Motion control commands that result in
preservationoftheminimumstabilityrequirement
arenotprovided(L‐2).
4.1.3 Step1.2:Hierarchicalcontrolstructure
Figure2presentsthehierarchicalcontrolstructure
for the system of study. Each box indicates a
controller.Controlactionsare
showninred.Feedback
isshowninblue.
The hierarchical control structure displays the
relationships between various controllers of the
system. As an example of the control hierarchy, we
describetheflow of commandsfor a typicalevasive
maneuver: The AIS transceiver provides target ship
(TS)informationtothemotionpredictor.
Basedonan
analysis of the OS and TS trajectories, CADCAs are
retrievedforevasivemaneuversattheship’scurrent
speedandforvariousrudderangles(e.g.5°,10°,35°).
These are restricted by the maximum allowable
rudderangleprovidedbythestabilitycomputer.The
CADCAsarethenprovidedto
theDSSanddisplayed
totheOOW.Toperformtheevasiveaction,theOOW
provides the rudder command signal to the ship’s
rudder. The rudder then imparts a turning force on
theship.
Figure2.HierarchicalcontrolstructureoftheDSS.
4.1.4 Step1.3:IdentifyUnsafeControlActions(UCAs)
The subsystem of focus for the next two steps of
the STPA is the stability computer and its singular
control action: “provide rudder angle limits to the
conflictresolver”.Rudderanglelimitsaredetermined
by considering the intact stability estimate provided
by the
OOW at the beginning of the voyage, and
dynamic estimates of wave forecast and direction
retrieved from the GFS server at periodic intervals.
Onevariablethatcannotbemeasuredbythesystemis
thetruevalueoftheship’sintactstability.Instead,the
system relies upon the estimate of the
loading
conditionprovidedbytheOOW.
Table 1 presents the identified unsafe control
actions for the stability computer. Based on the
estimate of intact stability and the environmental
conditionsreceived,thestabilitycomputerdecidesthe
maximum possible rudder angles for an evasive
maneuver. Evasive maneuvers with higher rudder
angleswillbe
consideredforhighintactstabilityand
low wave height. Conversely, the range of rudder
anglesdecreasesforestimatesoflowintactinstability
andforecastswithhighwaveheight.
379
Table1.IdentifiedUCAsofthestabilitycomputer
________________________________________________
Notproviding Providingcauses Tooearly, Stoppedtoosoon,
causeshazard hazard toolate, appliedtoolong
outoforder
________________________________________________
UCA‐1: UCA‐2:Stability UCA‐3: UCA‐4:Stability
Stability computerStability compu terremoves
computer provides computer rudderanglelimits
doesnot excessivelylenientprovides duringevasive
provide rudderangle proper maneuver[H‐2.1].
rudder limits[H‐1.1, rudder
angle H‐2.2]. angle
limits[H‐2.2].limitsafter
evasive
maneuver
isinitiated
[H‐1.1,H‐2.1].
UCA‐5:StabilityUCA‐6:Stability
computerprovidescomputer
excessivelystrictmaintainsrudder
rudderangleanglelimitsafter
limits[H‐1.2,H‐2.1].executionof
evasivemaneuver
[H‐1.1].
________________________________________________
4.1.5 Step1.4:IdentifyLossScenarios(LSs)
Loss scenarios (LS) are generated to determine
causalfailuresthatcanleadtounsafecontrolactions.
For conciseness, we focus on UCA‐2: “Stability
computer provides excessively lenient rudder angle
limits[H‐1.1,H‐2.2].”Twoexamplelossscenariosare
presentedbelow.
LS‐
2‐1: Wave forecast information underpredicts
the significant wave height. For the predicted wave
height, the stability computer underpredicts the
expectedrollmotionoftheship,andprovideslenient
rudder angles limits. This allows for performing an
evasive maneuver at a rudder angle that leads to
excessiverollmotionofthe
vessel.[H‐2.2].
LS‐2‐2:Atthebeginningofthevoyage,theOOW
provides the weight and location estimate which
resultsinanoverpredictedmeasureofintactstability.
Fortheestimatedshipstabilitycondition,thestability
computerprovideslenientrudderanglelimitsforthe
given wave forecast information. This
leads to
excessiverollmotionofthevessel[H‐2.2].
Loss scenarios provide context for developing
performance tests of system’s functions. For LS‐2‐1,
weidentifythat theaccuracyofforecastinformation
impacts the ability of the controller to provide
adequate rudder angle limits. LS‐2‐2 identifies the
presence
ofanunmeasuredPVwithinthesystem:the
ship’s actual stability. The difference between the
ship’s actual and predicted intact stability may
contributetoalossofthevessel.
4.2 Step2:IntegrationTesting
4.2.1 Step2.1:FunctionalTesting
The first step is the identification of the PVs that
contribute
to the unsafe control action. For UCA‐2:
“Stability computer provides excessively lenient
rudder angle limits [H‐1.1, H‐2.2]”, the variables of
interest are the target ship presence (PV‐1), rudder
anglelimits(PV‐2),predictedshipintactstability(PV‐
3), and forecasted wave height (PV‐4). Furthermore,
twovariables
existbutareunobservedbythesystem:
the observed intact stability and wave height.
Therefore,PVs‐3and‐4arerevised:theseareinstead
presented as the differences between the observed
andpredictedintactstability(PV‐3)andwaveheight
(PV‐4).
The functional test is developed for the
requirement
ofthesystemtoprovideadequaterudder
anglelimits.
FunctionalTest:Adequaterudderanglelimitsare
correctlyprovidedduringoperation.
Objective:Toassesstheprovisionofrudderangle
limitsduringsystemoperation.
Setup: Utilize simulation test‐bed. The test‐bed
should include a six degree‐of‐freedom
hydrodynamicmodel
oftheshipwithitsactuators
in addition to a world model that can consider
waves and wind forces. Review code for
generatingrudderanglelimitsfromlookuptables.
Execution: Initiate sailing with a given forward
speed, given sea‐state, and a planned obstacle
conflict. Evaluate rudder angle provision
for all
expected stability conditions (PV‐3) and wave
heights (PV‐4). Repeat for various obstacle
configurations(PV‐1).
Evaluation: Review the provided rudder angle
limits by the stability computer. Ensure that
rudderanglelimitsareupdatedwhennewforecast
information is retrieved by the system. System
should recognize latent
information and display
information through DSS that wave forecast
information is delayed. Observe failure tolerant
behavior and appropriate display information on
theDSSinterface.
4.2.2 Step2.2:PerformanceTesting
TheperformancetestisdevelopedforLS‐2‐2.The
evaluation of performance is derived from the
assessmentofPVsthat
contributetotheLS.ThePVof
primary interest is the difference in predicted and
actualshipstability(PV‐3).
Performance Test: Analyze rudder angle limit
provisionbehaviorforover‐andunder‐predictedship
intactstability(LS‐2‐2).
Objective:Quantifythebehaviorofthesystemfor
under estimates and
over‐estimates of the ship’s
intactstability.
Setup: Utilize simulation test‐bed. As before, the
test‐bed should include the hydrodynamic model
oftheshipandactuators.Thefidelityoftheworld
model should be reduced to allow for faster
computation. Review code for generating rudder
anglelimitsfrom
lookuptables.
Execution: Initiate sailing with a given forward
speed, planned obstacle conflict, and ratio of
estimatedtoactualshipintactstability(i.e.,ratios
greaterthanoneindicateover‐prediction).Repeat
for range of ratios. Repeat for various sea‐states,
forwardspeeds,andtimedelays.
Evaluation: Quantify the ratio
of rudder angle
limitsfortheestimatedandactualintactstability.
Investigate the necessity of safety margins to
increase the rudder angle limits for various
forwardsailingspeeds.
380
4.2.3 Useofintegrationtests
Thetestcasespresentedherereflectanapplication
of the methodology to a section of the system of
study.Alargerscaleanalysiswouldidentifya larger
number of tests to assess system functionality and
performance.
5 DISCUSSION
5.1 Methodologicalimplications
The approach demonstrates the
development of
integration testing for system verification. The
development of the hierarchical control structure
allows for the visualization of control and feedback
forsystembehavioranalysis.Modelingthesystemas
acontrolsystemensuresafocusonsystembehavior,
and the results of STPA steps 3 and 4 reveal
interactions
among components that are not
immediately apparent. Furthermore, the interactions
are modeled across human, hardware, and software
aspects of system design. The method allows for
determiningkey variablesthatshouldbeinvestigated
andtestedduringintegrationtesting.
Performing STPA at an early stage of system
development can identify key aspects of
integration
well in advance of the verification process. The
traditionalV‐modeldelaysverificationuntilafterthe
coding has beenfinalized.However, designchanges
towardsthebeginningofaprojectareoftenlesscostly
thanthoseimplementedtowardstheend.
The use of digital testing requires additional
considerations that are
not explored here in detail.
Theuseofahydrodynamicmodelrequiresextensive
development,andvalidationofthemodelshouldbe
achieved using full‐scale data for the vessel, if
possible[25].
5.2 Limitationsofthestudy
Theapproachrequiresbackgroundknowledgeonthe
system’s purpose, structure, and operation.
Furthermore,STPA
isalaborintensivemethodology.
NewresearchhasstudiedthesynthesisofSTPAwith
systematicmethodsfortestgeneration.Theseinclude
automatic scenario generation [26] and conformance
and fault injection (CoFI) [27]. For the purpose of
integration,suchmethodscouldbedirectedtowards
theresultofaninput/outputanalysisfor
acontroller.
5.3 Recommendationsforfutureresearch
Theapproachcomplementssimulationasamethodof
verification for autonomous systems. Future work
could describe the test‐bed and setup of the
simulation test‐bed. The research can be further
extendedtoincludeananalysisofthetestingresults.
Additional work would focus
on how the results of
the simulations impact the system’s operation, and
describeanymodificationstothesystemstructure.If
the results of testing are found to improve system
design, the methodology should be iterative to
incorporatechangestosystemstructureandbehavior.
Forexample,theintroductionofasystem
toestimate
the vessel’s intact stability [28] would have to be
modeledintherevisedcontrolhierarchy.Theanalysis
should be modified to include the effects of any
design changes, as they could potentially introduce
emergentbehaviortothesystem.
6 CONCLUSION
This paper has demonstrated the use of STPA to
analyze system behavior and identify test cases for
system operation. Requirements for verification of
critical systems fall into two categories: failure
handling and integration testing. The method was
applied to a decision support system for collision
avoidance focusing specifically on stability
monitoring for collision avoidance. The hierarchical
control structure demonstrates
the relationships
betweencontrollers,andtheUCAsandLSswereused
to suggest functional and performance integration
testsforthestabilitycomputerasacasestudy.
STPA is uniquely positioned to analyze the
hazardsrelated tointegrationofcomplex systemsof
systems. Modeling the system as a control structure
presentsa
structuredapproachtoidentifyinteractions
amongcontrollers.Furthermore,duetoitsflexibility,
itcanbeusedatearlystagesofsystemdevelopment.
Theuseofthemethodcanleadtomorerobustdesign
ofsafetycriticalsystems.
ACKNOWLEDGMENTS
The work has been performed as part of the Detection,
prediction, and solutions for safe operations of MASS
(ENDURE) project (number
NOR/POLNOR/ENDURE/0019/2019–00), supported by the
PolishNational centre for Research and Development and
financed by Research Council of Norway. Research
supported in part by the Research Council of Norway
throughSFI
Autoship(RCN309230).
REFERENCES
[1]M. Gil, K. Wrobel, J. Montewka, F. Goerlandt, A
bibliometric analysis ´ and systematic review of
shipboard Decision Support Systems for accident
prevention, Safety Science 128 (2020) 104717.
doi:10.1016/j.ssci.2020.104717.
[2]IEEE, IEEE Standard for System, Software, and
HardwareVerificationandValidation,TechnicalReport,
2017.ConferenceName:IEEEStd1012‐2016
(Revisionof
IEEE Std 1012‐2012/ Incorporates IEEE Std 1012‐
2016/Cor1‐2017).
[3]DNV, Rules for Classification: Ships, Technical Report
DNV‐RU‐SHIP,2022.
[4]DNV, Class Guideline: Autonomous and remotely
operated ships, Technical Report DNV‐CG‐0264, DNV,
2021.
[5]R. Skjetne, O. Egeland, Hardware‐in‐the‐loop testing
of
marine control system, Modeling, Identification and
Control:ANorwegianResearchBulletin27(2006)239–
258.doi:10.4173/mic.2006.4.3.
[6]O.Smogeli,J.E.Skogdalen,ThirdPartyHILTestingof
Safety Critical Control System Software on Ships and
Rigs,OnePetro,2011.doi:10.4043/22018‐MS.
381
[7]T.A.Pedersen,J.A.Glomsrud,E.‐L.Ruud,A.Simonsen,
J. Sandrib, B.‐O. H. Eriksen, Towards simulation‐based
verification of autonomous navigation systems, Safety
Science129(2020)104799.doi:10.1016/j.ssci.2020.104799.
[8]K. Wrobel, J. Montewka, P. Kujala, Towards the
development of a system‐theoretic model for safety
assessment
ofautonomousmerchantvessels,Reliability
Engineering & System Safety 178 (2018) 209–224.
doi:10.1016/j.ress.2018.05.019.
[9]N. Leveson, J. Thomas, STPA Handbook,
https://psas.scripts.mit.edu/home/get
file.php?name=STPAhandbook.pdf,2018.
[10]R.Patriarca,M.Chatzimichailidou,N.Karanikas,G.Di
Gravio, The past and present of System‐Theoretic
Accident Model And Processes (STAMP) and its
associatedtechniques:
Ascopingreview,SafetyScience
146(2022)105566.doi:10.1016/j.ssci.2021.105566.
[11]N. A. Zikrullah, H. Kim, M. J. van der Meulen, G.
Skofteland,M.A.Lundteigen,A comparisonofhazard
analysis methods capability for safety requirements
generation,ProceedingsoftheInstitutionofMechanical
Engineers, Part O: Journal of Risk and Reliability 235
(2021) 1132–1153. doi:10.1177/1748006X211003463,
publisher:SAGEPublications.
[12]R. Yang, I. B.Utne, Towardsan online risk modelfor
autonomousmarinesystems(AMS),OceanEngineering
251(2022)111100.doi:10.1016/j.oceaneng.2022.111100.
[13]B.Rokseth,I.B.Utne,J.E.Vinnem,Asystemsapproach
to risk analysis of maritimeoperations, Proceedings of
the
InstitutionofMechanicalEngineers,PartO:Journal
of Risk and Reliability 231 (2017) 53–68.
doi:10.1177/1748006X16682606.
[14]B. Rokseth, I. B. Utne, J. E. Vinnem, Deriving
verification objectives and scenarios for maritime
systems using the systems‐theoretic process analysis,
Reliability Engineering & System Safety 169 (2018) 18–
31.doi:10.1016/j.ress.2017.07.015.
[15]B.Rokseth,
O.I.Haugen,I.B.Utne,SafetyVerification
forAutonomousShips,MATECWebofConferences273
(2019) 02002. doi:10.1051/ matecconf/201927302002,
publisher:EDPSciences.
[16]N. Leveson, Engineering a Safer World: Systems
Thinking Applied to Safety, Engineering systems, MIT
Press,Cambridge,Mass,2011.
[17]NTSB, Capsizing of Roll‐on/Roll‐off Vehicle Carrier
Golden Ray, St. Simons Sound, Brunswick River, near
Brunswick,Georgia,September8,2019(2020).
[18]H.Kim,S.Haugen,I.B.Utne,Assessmentofaccident
theoriesformajoraccidentsfocusingontheMVSEWOL
disaster: Similarities, differences, and discussion for a
combined approach, Safety Science 82 (2016) 410– 420.
doi:10.1016/j.ssci.2015.10.009.
[19]E. V. Lewis, Principles of naval architecture, 2nd
revision (3rd ed.) ed., Society of Naval Architects and
Marine Engineers, Jersey City, 1988. OCLC:
ocm37002765.
[20]P.Krata,T.Hinz,S.A.Dugan,M.Marley,J.Montewka,
Prediction and Evaluation of an Angle of Heel due to
TurningManeuverofSmall
TrainingShips:Comparison
of Dynamic Analysis and Static Design Criteria, in:
Proceedings of the 15th International Symposium on
PracticalDesignofShipsandOtherFloatingStructures,
2022.
[21]J. Montewka, P. Krata, T. Hinz, M. Gil, K. Wrobel,
Probabilistic model estimating the expected maximum
rollangleforavesselin
theturn(2022)10.
[22]K.Wrobel,M.Gil,P.Krata,K.Olszewski,J.Montewka,
Ontheuse´ofleadingsafetyindicatorsinmaritimeand
theirfeasibilityforMaritimeAutonomousSurfaceShips,
ProceedingsoftheInstitutionof Mechanical Engineers,
Part O: Journal of Risk and Reliability (2021)
1748006X211027689. doi:10.1177/1748006X211027689,
publisher:
SAGEPublications.
[23]M. Gil, J. Montewka, P. Krata, T. Hinz, S. Hirdaris,
Determinationofthedynamiccriticalmaneuveringarea
in an encounter between two vessels: Operation with
negligibleenvironmentaldisruption,OceanEngineering
213(2020)107709.doi:10.1016/j.oceaneng.2020.107709.
[24]M.Gil,Aconceptofcriticalsafetyareaapplicableforan
obstacleavoidance
processformannedandautonomous
ships,ReliabilityEngineering&SystemSafety214(2021)
107806.doi:10.1016/j.ress.2021.107806.
[25]K. H. Chua, S. Coutinho, A. Norahim, D. Konovessis,
DevelopmentofRecommendationsforDigitalTestingof
MASSNavigation Safety prior to Sea Trials, Journal of
Physics: Conference Series 2311 (2022) 012025.
doi:10.1088/1742‐6596/2311/1/012025.
[26]
T. A. Pedersen, A. Neverlien, J. A. Glomsrud, I.
Ibrahim,S.M.Mo,M.Rindarøy,T.Torben,B.Rokseth,
Evolution of Safety in Marine Systems: From System‐
TheoreticProcessAnalysis to Automated Test Scenario
Generation, Journal of Physics: Conference Series 2311
(2022)012016.doi:10.1088/1742‐6596/2311/1/012016.
[27]C.M.Hirata,A.
M.Ambrosio,CombiningSTPAWith
CoFI to Generate Requirements and Test Cases for
Safety‐Critical System, IEEE Systems Journal 16 (2022)
6635–6646. doi:10.1109/JSYST.2022.3200586, conference
Name:IEEESystemsJournal.
[28]L.SantiagoCaamano,M.MiguezGonzalez,S.Allegue
Garcia, V. Diaz Casas, Evaluation of onboard stability
assessment techniques under real operational
conditions,
Ocean Engineering 258 (2022) 111841.
doi:10.1016/j.oceaneng.2022.111841.
