789

1 INTRODUCTION

Autonomy and artificial intelligence are disrupting

many sectors, including the marine industry. Many

companies and academia are researching to evolve

the field. Some companies have even started testing

autonomy in real commercial routes (with safety

drivers on board to meet current regulations). In late

2018 a ferry, developed by Finferries and Rolls Royce,

went between two cities in Finland, first navigating

autonomously and then remotely operated when

returning [1]. In Norway, also in 2018, Kongsberg

started testing autonomy on an autonomous ferry

with passengers and cars on board, mainly to reduce

the workload and to increase the safety [2]. To

convince authorities to change regulations to permit

using ships without a crew on board, it is of utmost

importance to guarantee safety. A human onboard a

ship is very flexible, and will in many situations

discover if the ship is behaving strangely or if an

unexpected event arises. When removing the crew,

the vessel will need to incorporate this extra safety

feature into the system instead.

When it comes to safe navigation, to have a correct

position is vital. Nowadays, crew members rely

heavily on the Global Positioning System (GPS) for

this. A loss of the GPS signal, or a jammed or spoofed

GPS, can for a crew-less ship result in hazardous

situations. The global quality assurance and risk

management company DNV GL believes unmanned

ships may need alternative positioning methods to

convince authorities that their safety is satisfactory

[3]. Furthermore, they believe autonomous ships will

not be fully autonomous for many years, but instead

rely on autonomy and remote control in combination.

Rolls Royce also believes this, as they see the

teleoperation of ships as a key technology in the

transferring process towards autonomous ships [4].

Moreover, they claim that the teleoperation of an

autonomous vessel will increase reliability and

performance. The communication link for the

teleoperation system is vulnerable to downtime,

VR Teleoperation to support a GPS-free Positioning

System in a Marine Environment

M. Lager, E.A. Topp & J.Malec

Lund University, Lund, Sweden

ABSTRACT: Small autonomous surface vehicles (ASV) will need both teleoperation support and redundant

positioning technology to comply with expected future regulations. When at sea, they are limited by a satellite

communication link with low throughput. We have designed and implemented a graphical user interface (GUI)

for teleoperation using a communication link with low throughput, and one positioning system, independent of

the Global Positioning System (GPS), supported by the teleoperation tool. We conducted a user study (N=16),

using real-world data from a field trial, to validate our approach, and to compare two variants of the graphical

user interface (GUI). The users experienced that the tool gives a good overview, and despite the connection with

the low throughput, they managed through the GUI to significantly improve the positioning accuracy.

http://www.transnav.eu

the International Journal

on Marine Navigation

and Safety of Sea Transportation

Volume 14

Number 4

December 2020

DOI: 10.12716/1001.14.04.01

790

though, and during this time, the system must solve

the actions autonomously.

The work described by this paper has focused on

how to use remote operation to improve positioning

accuracy for small affordable vessels. Unmanned

ground vehicles (UGV) have, for many years, been

teleoperated to master harsh environments during,

e.g., military or search and rescue (SAR) missions [5]–

[7]. Small autonomous vessels at sea are also

essential, and Murphy believes they will play an

important role during future SAR operations [8]. The

challenges with remote control and positioning are

similar for small and large ships. However, the

communication link’s throughput sets a limitation on

smaller, more affordable vessels, as they can not have

a large satellite antenna due to the size, weight, and

cost constraints. This limitation makes the streaming

of video and transmission of high-resolution images

infeasible. For the positioning problem, we have, for

the same reason, confined ourselves only to use

affordable navigation sensors.

Figure 1. A participant of the user study taking a bearing

by pointing towards an augmented landmark.

The positioning system is built upon our previous

implementation with terrain-aided navigation (TAN),

presented in [9]. This paper estimated the position

from a real-world field trial by comparing the bottom

depth and magnetic intensity with available maps. To

enhance the position accuracy even further, we

manually measured bearings to landmarks from the

recorded 360 image, making it possible for the

positioning tool to adjust the position estimation

accordingly. This is not possible to do manually on an

unmanned ship. In this new work, a user instead

measures these bearings from a teleoperation system

in virtual reality (VR), see Figure 1.

The teleoperation system also builds on our

previous work, presented in [10], [11]. This work

focused on developing a teleoperation tool with a

low-cognitive load that could provide a good

situational awareness (SA), leading to better safety for

the vessel. In the work described in the latter paper,

we developed a specific GUI to compare the

performance when using VR, 3D visualization on a

laptop, and 2D visualization on a laptop. In this

earlier study, we observed that the longer available

time for decisions at sea, measured in seconds or

minutes, makes it ideal for teleoperation. This

contrasts with the fast dynamics of the traffic

situations for cars and airplanes, often measured in

milliseconds, reported as challenging teleoperation

areas due to the vulnerability from mainly long

latency [12], [13]. Several research papers propose

methods to compensate or predict the teleoperated

vehicle’s pose to mitigate the latency problem [14]–

[16]. We use this knowledge to predict our current

position based on heading, speed, and