Instrumentation & Measurement Magazine 24-2 - 118

Responding to Illegal Activities
Along the Canadian Coastlines
Using Reinforcement Learning
Mohammed Abouheaf, Shuzheng Qu, Wail Gueaieb, Rami Abielmona, and Moufid Harb

achine learning (ML) algorithms can prove
to be instrumental in certain complex ill-conditioned systems when inserted as a middle
layer to interface low-level hardware, such as sensors and
actuators, and high-level decision-making kernels. Such an
interface provides a secondary, or supervisory, conditioning layer that would enhance the system's robustness in
the face of various types of uncertainties and disturbances.
This article elaborates how ML can leverage the solution of
a contemporary problem related to the security of maritime
domains.
The worldwide " Illegal, Unreported, and Unregulated "
(IUU) fishing incidents have led to serious environmental and
economic consequences which involve drastic changes in our
ecosystems in addition to financial losses caused by the depletion of natural resources. The term " illegal " refers either
to contravening the fishing regulations or conducting fishing activities unlawfully under the jurisdiction of a certain
maritime territory. In this context, " unreported " indicates the
failure to report to the relevant authorities, while " unregulated " describes fishing activities in areas with no applicable
measures to control the catch. The Fisheries and Aquatic Department (FAD) of the United Nation's Food and Agriculture
Organization (FAO) issued a report which indicated that the
annual losses due to IUU fishing reached $25 Billion [1]. This
imposes negative impacts on the future-biodiversity of the marine ecosystem and domestic Gross National Product (GNP).
Maritime Domain Awareness (MDA) is concerned with the situational awareness relevant to the activities that have direct
or indirect impacts on the operational, organizational, economical, and safety sides of the maritime domains [2], [3]. The
assets in MDA support different missions, such as monitoring,
risk assessment, and management. Hence, robust interception mechanisms are increasingly needed for detecting and
pursuing the unrelenting illegal fishing incidents in maritime
territories.

Decision-Making System
The IUU fishing problem can be broken down into at least
three sub-problems:
1. Recognizing an IUU activity, where radar and satellite
monitoring systems are deployed to detect the unreported event and provide live information about its
status, such as the vessels' location measurements, for
instance.
2. Deciding on the resources (coast guard vessels, military
planes, etc.) to dispatch in the pursuit of the IUU vessels.
The decision may depend on many factors, including the
locations of the pursuing vehicles, their speeds, cost of
operation, and fuel consumption, to name a few. Herein,
autonomous sea vessels are considered to pursuit the
illegal shipping activity.
3. Coordinating the motion of the fleet of pursuers to catch
the IUU vessel while still in local waters.
In this article, only the latter task is addressed. It is formulated
as a pursuer-evader problem that is tackled within an ML framework. One or more pursuers, such as law enforcement vessels,
intercept an evader (i.e., the illegal fishing ship) using an online
reinforcement learning mechanism that is based on a value iteration process. It employs real-time navigation measurements of
the evader ship as well as those of the pursuing vessels and returns back model-free interception strategies. Means of adaptive
critics are employed to approximate the learning process using
actor-critic neural networks, where a gradient descent approach
is followed to tune the weights of the neural networks. Test cases
are simulated with the aid of Total::Perception™ (T::P) engine, a
professional predictive analytics decision-making software that
is developed to mimic the situational awareness in maritime as
well as other domains [4], [5].

Machine Learning Literature
The significant advances in the architectures of digital processing units enabled the implementation of sophisticated

This work was partially supported by NSERC Grant EGP 537568-2018.
118

IEEE Instrumentation & Measurement Magazine
1094-6969/21/$25.00©2021IEEE

April 2021

Instrumentation & Measurement Magazine 24-2

Table of Contents for the Digital Edition of Instrumentation & Measurement Magazine 24-2