527
1 INTRODUCTION
Ever since the 1982 United Nations Convention on the
Law of the Sea (UNCLOS) entered into force in 1994,
the rights and duties of coastal, port and flag States in
respect of principal maritime zones, namely the
territorial sea, the exclusive economic zone (EEZ) and
the high seas became the bases and driving forces for
the rapid changes in the maritime management.
Systems or schemes have been introduced to enhance
the safety of life at sea, the environment protection,
and maritime security.
As for the fisheries, besides the provisions of
UNCLOS, the rapid depletion of key fish stocks has
made it imperative that governments achieve greater
control over fishing activities. In order to ensure
sustainable fisheries, a mechanism called monitoring,
control and surveillance (MCS) was introduced for
implementation of agreed policies, plans or strategies
for oceans and fisheries management [1].
Fisheries MCS can be defined as follows:
− “Monitoring” includes the collection,
measurement and analysis of fishing activities
including, but not limited to: catch, species
composition, fishing effort, by-catch, discards, area
of operations, etc.
− “Control” involves the specification of the terms
and conditions under which resources can be
harvested.
− “Surveillance” involves the regulation and
supervision of fishing activity to ensure the
national legislation and terms, conditions of
access, and management measures are observed.
Flag States have the responsibility to know where
their vessels are located. Flag States must also have
some means of determining what each of their fishing
A Study of Correlation between Fishing Activity and AIS
Data by Deep Learning
K
.Y. Shen, Y.J. Chu, S.J. Chang & S.M. Chang
National Taiwan Ocean University, Keelung, Taiwan
ABSTRACT: Previous researches on the prediction of fishing activities mainly rely on the speed over ground
(SOG) as the referential attribute to determine whether the vessel is navigating or in fishing operation. Since
more and more fishing vessels install Automatic Identification System (AIS) either voluntarily or under
regulatory requirement, data collected from AIS in real time provide more attributes than SOG which may be
utilized to improve the prediction. To be specific, the ships' trajectory patterns and the changes in course
become available and should be considered.
This paper aims to improve the accuracy in the identification of fishing activities. First, we do feature extraction
from the AIS data of coastal waters around Taiwan and build a Recurrent Neural Network (RNN) model. Then,
the activity data of fishing vessels are divided into fishing and non-fishing. Finally, based on the testing by
feeding various fishing activity data, we can identify the fishing status automatically.
http://www.transnav.eu
the
International Journal
on Marine Navigation
and Safety of Sea Transportation
Volume 14
Number 3
Septemb
er 2020
DOI:
10.12716/1001.14.03.01
528
vessels is catching. Although the data to be reported
will vary from fishery to fishery, flag States should
require their fishing vessels to report timely, complete
and accurate information concerning fishery
activities, including: vessel identification, position,
course, speed, fishing effort, catch composition, zone
entry/exit (including closed areas entry/exit). Flag
States should also establish a mechanism to verify the
accuracy of reported data and should penalize the
failure to report and misreporting of data. For serious
offences, such sanctions should include withdrawal or
suspension of the vessel’s authorization to fish.
The term “illegal, unreported and unregulated
fishing” or IUU fishing is used to describe a wide
range of irresponsible fishing activities, such as
reflagging of fishing vessels to evade controls, fishing
in areas without authorization, failure to report or
misreporting catches. Such activities undermine
efforts to manage marine fisheries properly and
impedes progress toward the goal of sustainable
fisheries.
Apparently, automatic detection and identification
of fishing activities is essential to effective fishery
MCS and sustainable fishery. This is the focus and
main purpose of the work presented in this paper. It
is envisioned that development of such functionality
can further contribute to maritime spatial planning as
well as maritime safety and security.
One of the most efficient and cost-effective tool for
fisheries MCS is Vessel Monitoring System (VMS).
Over the past 20 years, a growing number of States
have introduced VMS requirements for their fishing
vessels or as a condition of access for foreign vessels
to fish in waters under their jurisdiction. Most
international agreements adopted by regional
fisheries management organizations (RFMOs) also
require VMS.
In the early days of fishing activity detection, most
researchers use data collected by VMS to predict
when the vessels is in fishing operation. VMS mainly
relies on satellite-based automatic location
communicators, including Inmarsat-C, ARGOS, and
Iridium, and the position report interval is usually set
at 1 hour for coastal monitoring due to the cost.
The vessel’s speed is used as a threshold to judge
the behavior [2,3,4]. However only the trawling
accuracy is relatively high when compared with other
fishing methods. In order to improve the accuracy,
Artificial neural networks(ANN) are used for
analyzing the VMS data, and the optimization of the
parameters is adjusted by sensitivity method [5,6].
Compared with VMS, Automatic Identification
System (AIS) provides much detailed locations and
more attributes of the vessels in real-time. Besides,
AIS position reports are broadcast in maritime VHF
band using standard unencrypted message formats,
which can be collected by coastal receivers in range.
AIS data can even be received by satellites, thus called
Satellite AIS (S-AIS). S-AIS can cover deep sea fishing
area, although with some data loss and latency. AIS
data has become an important asset to researches on
vessel tracks and fishing behaviors, e.g. [7]. In [8],
machine learning is used to identify the three type of
fishing activity, i.e. trawler, longliner, and purse
seiner, from S-AIS data and label the points as fishing
or non-fishing. Because longline fishing is a
complicated fishing method, in [9] a novel approach is
proposed for identifying fishing activity using the
Conditional Random Fields. In [10], deep learning is
used with auto-encoders to automatically find fishing
features. However, the research in [10] is using S-AIS
data to detect fishing activity of distant water fishing.
So far in the literature, to the author’s knowledge,
none of the AIS-based fishing activity detection is for
small and medium-sized fishing vessels on coastal
waters.
To detect fishing activity and improve
identification performance, we implement an
identification methodology based on deep learning.
Key features of fishing are created in advance and a
multi-layered bidirectional long short term memory
model is built to predict three types of fishing
activities, namely trawling, trolling, and longline
fishing, on coastal waters around Taiwan. This paper
is organized as follows. Section II introduces
terminologies used throughout this paper. Section III
describes the data preprocessing and reports the
results of the experiments. Conclusions are then
presented in Section IV.
2 BACKGROUND OF METHODS
2.1 Recurrent Neural Network
Recurrent Neural Network (RNN) is a well-known
model to deal with sequential data. The structure of a
simple RNN, illustrated in Fig. 1, has feedback loops
which let model maintain memory over time. This
means input has not only the result of the previous
hidden layer, but also the value predicted at the
previous time.
An RNN can be described mathematically as
follows. Given a sequence of feature vector
T
X
=
{
1
x
,
2
x
, … ,
T
x
}. An RNN with a hidden vector
sequence
T
H
= {
1
h
,
2
h
, … ,
T
h
} and output vector
sequence
T
Y
= {
1
y
,
2
y
, …,
T
y
} is calculated as
follows:
( )
1 11t h t ht
h Wx Wh b
σ
−
= ++
(1)
( )
22ty t
y Wh b
σ
= +
(2)
where
i
W
and
i
b
denote the input weight matrix
and bias vector, respectively.
h
W
denotes the weight
matrix between consecutive hidden states (
1
t
h
−
and
t
h
), while
h
σ
and
y
σ
denote activation functions of
the hidden layer and output layer.
Figure 1. Recurrent Neural Network architecture
529
2.2 Bidirectional Recurrent Neural Network
A bidirectional RNN (BRNN), illustrated in Fig. 2,
consists of two separate hidden layers that both
connect to the same input and output. The first layer
learns from the previous time steps and the second
layer learns from the following time steps. Therefore,
BRNN can exploit information both from the past and
the future.
Figure 2. Bidirectional recurrent neural network architecture
2.3 Long Short-Term Memory
When the sequence is long enough, RNNs suffer from
the vanishing gradient problem. Therefore, RNN is
able to remember only short-term memory sequences.
To solve this problem, a variant of RNN called Long
Short-Term Memory (LSTM) was proposed by
Hochreiter & Schmidhuber[11]. The only different
component between LSTM architecture and RNN
architecture is the memory cell. As illustrated in Fig.
3, there are three gates in an LSTM cell, including an
input gate, a forget gate, and an output gate, denoted
as
t
i
,
t
f
and
t
o
respectively. Each gate has a value
between 0 and 1. The value 0 means that the gate is
closed, while the value 1 means that the gate is
opened. In an LSTM layer, the hidden layer output
t
h
in Eq.1 is replaced by the following equations:
(
)
11
ii
t xt ht
i Wx Wh b
σ
−
= ++
(3)
(
)
12
ff
t xt ht
f Wx Wh b
σ
−
= ++
(4)
(
)
t t1 t 1 3
cc
t xt ht
c f c i tanh W x W h b
−−
=+ ++
(5)
( )
14
oo
t xt ht
o Wx Wh b
σ
−
= ++
(6)
( )
t
tanh
tt
ho c=
(7)
where
t
c
is the memory cell, superscripts i, f, o, c are
the input gate, forget gate, output gate and input cell
state, respectively.
Figure 3. Long shor-term memory cell
3 EXPERIMENTS AND RESULTS
3.1 Data Pre-processing
− Data Cleaning: In this experiment, we use AIS data
of class B shipborne stations and sort the data in
order according to Maritime Mobile Service
Identity (MMSI) and timestamps. AIS data are
subject to the integrity, availability and accuracy of
inputs from interfaced shipborne sensors as well
as configuration settings and manual entries [12].
Hence data pre-processing needs to be performed,
e.g. to remove repetitive data points or duplicate
MMSIs and outliers.
− Feature Selection: The following three features are
not considered suitable for use:
− Ship Size or Tonnage: It affects the speed of
fishing. Tonnage may be obtained separately from
VMS database or inferred indirectly from the ship
length reported in AIS data. However, in coastal
waters around Taiwan, the ship length
information in AIS data of fishing vessels are
either unavailable or unreliable.
− Heading: Almost all coastal Taiwanese fishing
vessels do not have gyrocompass installed and
connected to AIS, thus the heading attribute is
basically unavailable.
− Latitude and Longitude: Using absolute latitude
and longitude values of the positions leads to
overfitting, which may make the prediction
accurate only in that specific area. It is more
appropriate to use the calculated relative positions
between consecutive points.
Since fishing activity is highly correlated with the
ship’s speed and the change of ship’s course, the
Speed over Ground (SOG) and change in Course over
Ground (COG) are indispensable features. According
to our observation, the characteristics in operation
distance and operation time differ in each type of
fishing method. SOG is the instantaneous speed,
which may cause some errors when used in judging
long-term behavior. In order to improve fishing
activity detection, we not only consider the SOG and
change of COG, but also calculate the differential time
(
T∆
), relative distance (
D∆
) and average speed
(
V
avg
) between consecutive points, as shown in Fig.4.
530
Figure 4. Feature selection
3.2 Results
In this study, we build the multi-layer bidirectional
LSTM model with Keras, train three models by three
types of fishing activities and design three
experiments to compare the effect of feature selection.
Case 1 use one feature: SOG. Case 2 use three
features: SOG,
COG∆
and
V
avg
. Case 3 use five
features: SOG,
COG∆
,
V
avg
,
T
∆
and
D∆
. The
results are shown in TABLE I.
Table 1. Evaluation Using Different Features
_______________________________________________
Fishing Feature Selection Source Data
Gear Type Case 1. Case 2. Case 3. Track % of Fishing
Size Activity
_______________________________________________
Trawling 85.7% 95.9% 94.4% 33892 58.91%
Longlining 72.3% 86.4% 89.8% 39393 63.15%
Trolling 91.3% 99.1% 99.6% 84740 86.71%
_______________________________________________
TABLE I shows that considering only SOG is not
enough for the detection of trawling. Best
performance for trawling detection is obtained using
SOG,
COG∆
and
V
avg
. The longlining, as a
complicated fishing activity, is easily affected by the
depth of the longline setting. If the detection only
considers SOG, the accuracy is not good. After adding
COG∆
and
V
avg
, the prediction accuracy is raised
by 14.1%. By further adding
T∆
and
D∆
, the
accuracy is raised by another 3.5%. The accuracy of
trolling is already rather good when only SOG is
considered. When SOG,
COG∆
,
V
avg
,
T∆
and
D∆
are considered, the accuracy can be as high as
99.6%.
In general, Case 3 have better performance. Fig. 5,
Fig. 6 and Fig. 7 show the visualization of three
predicted results of Case 3 for different types of
fishing. Green point represents that it is actually
fishing and predicted to be fishing. Red point
represents that it is actually non-fishing and predicted
to be non-fishing. Blue point represents that it is
actually fishing but is predicted to be non-fishing.
Yellow point represents that it is actually non-fishing
but is predicted to be fishing.
Figure 5. The visualization of trawling detection results
Figure 6. The visualization of trolling detection results
Figure 7. The visualization of longlining detection results
Table 2. Detailed Performance Assessment of Case 3
__________________________________________________________________________________________________
Fishing Assessment of prediction
Gear Type Accuracy Sensitivity Specificity Positive Predictive Negative Predictive F1 score AUC
Value(PPV) Value(NPV)
__________________________________________________________________________________________________
Trawling 0.944 0.990 0.469 0.907 0.989 0.947 0.995
Longlining 0.898 0.840 0.442 0.988 0.806 0.908 0.981
Trolling 0.996 0.999 0.037 0.997 0.989 0.998 0.999
__________________________________________________________________________________________________
531
Detailed performance assessment of Case 3 are
shown in TABLE II. In order not to be affected by the
threshold, we use ROC curve (receiver operating
characteristic curve) and AUC (Area under the ROC
Curve) to assess the performance of the model, as
illustrates in Fig.8. The AUC values of three fishing
activity model are all exceeding 0.9, thus outstanding
discriminations are obtained.
Fgure 8. The ROC curve of Case 3.
4 CONCLUSIONS
This paper presents an approach to detect fishing
activities using multi-layered bidirectional LSTM
model for three main fishing types on the coastal
waters around Taiwan. Key features from AIS data
are found to raise the accuracy and verify their
influence on three models. For further research to
enhance the performance in fishing activity detection,
hybrid RNN model might be used to learn better
spatial representation and include auxiliary
information such as weather conditions or current
data.
ACKNOWLEDGMENT
Authors thank the Maritime and Port Bureau, Ministry of
Transportation and Communications for providing access to
the AIS data.
REFERENCES
[1] Flewwelling, P.,Cullinan, C., Balton, D., Sautter, R.P.,
Reynolds, J.E., "Recent trends in monitoring, control and
surveillance systems for capture fisheries", FAO
Fisheries Technical Paper. No. 415. Rome, FAO. 2002.
[2] Matthew J. Witt and Brendan J. Godley, “A step towards
seascape scale conservation: using vessel monitoring
systems (VMS) to map fishing activity,” PLoS One, 2(10),
e1111, 2007.
[3] S.J. Chang, “Satellite-based vessel tracking and
monitoring as the long-range mode of AIS,” Proceedings
of MTS/IEEE Oceans 2005 Conference, Washington D.C.,
USA, 2005.
[4] Rijnsdorp, A., Buys, A., Storbeck, F., and Visser, E,
“Micro-scale distribution of beam trawl effort in the
southern North Sea between 1993 and 1996 in relation to
the trawling frequency of the sea bed and the impact on
benthic organisms,” ICES Journal of Marine Science:
Journal du Conseil, 55(3), 403-419, 1998.
[5] Joo, R., Bertrand, S., Chaigneau, A., and Niquen,
M,“Optimization of an artificial neural network for
identifying fishing set positions from VMS data: an
example from the Peruvian anchovy purse seine
fishery,” Ecological Modelling, 222(4), 1048-1059, 2011.
[6] Russo, T., Parisi, A., Prorgi, M., Boccoli, F., Cignini, I.,
Tordoni, M., and Cataudella, S.,“When behaviour
reveals activity: Assigning fishing effort to métiers based
on VMS data using artificial neural networks,” Fisheries
Research, 111(1–2), 53-64, 2011.
[7] S.J. Chang, K.H. Yeh, G.D. Peng, S.M. Chang and C.H.
Huang, “From Safety to Security- pattern and anomaly
detections in maritime trajectories,” Proceedings of the
49th Annual International Carnahan Conference on
Security Technology, ICCST 2015.
[8] E. N. de Souza, K. Boerder, S. Matwin, and B. Worm,
“Improving fishing pattern detection from satellite ais
using data mining and machine learning,” PLOS ONE,
vol. 11, no. 7, p. e0158248, 2016.
[9] Baifan Hu, Xiang Jiang, and Stan Matwin, “Identifying
Fishing Activities from AIS Data with Conditional
Random Fields,” Federated Conference on Computer
Science and Information Systems (FedCSIS). IEEE, 2016,
pp. 1–7.
[10] X. Jiang, D. L. Silver, B. Hu, E. N. de Souza, and S.
Matwin, “Fishing activity detection from ais data using
autoencoders,” in Canadian Conference on Artificial
Intelligence. Springer, 2016, pp. 33–39.
[11] S. Hochreiter and J. Schmidhuber. Long short-term
memory. Neural computation, 9(8):1735–1780,1997.
[12] S.J Chang, “AIS Applications as an Efficient Tool for
VTS: Identifying and Coping with Discrepancy between
Ideal Cases, Standard and Real Situations,” Sea
Technology 47(3): 15-18, 2006.
