The Bridge - Issue 2, 2018 - 18

Feature

Distributed Stochastic Optimal Flocking
Control for Uncertain
Networked Multi-Agent
Systems
by: Hao Xu., Member, IEEE, and Wenxin Liu, Senior
Member, IEEE

ABSTRACT
This paper addresses the finite-horizon distributed
stochastic optimal flocking control design problem
for uncertain networked multi-agent systems
(MAS) in presence of network imperfections (e.g.
network-induced delays and packet dropouts) and
unknown disturbances. First, through adopting a
stochastic modeling technique, the networked MAS
dynamics are generated by effectively integrating
the effects from network imperfections. Moreover,
to handle the unknown disturbances, the networked
MAS is formulated as a two-player zero-sum
game, with control input and disturbance signal
acting as two players. Using the Neuro Dynamic
Programming (NDP) technique, a novel stochastic
actor-critic identifier scheme is proposed. The
proposed scheme can optimize the networked MAS
performance even when the system dynamics and
disturbances are completely unknown. Eventually,
simulation results demonstrate the effectiveness of
the proposed scheme.
Index Terms- networked multi-agent systems;
neuro dynamic programming (NDP); zero-sum
game

I. INTRODUCTION
FLOCKING is a behavior observed in nature by
which a large number of distributed interacting
entities or agents execute a common group

THE BRIDGE

objective, in a harmonic way, and without collisions.
Due to these characteristics, flocking techniques
are considered as a promising solution for solving
some of the challenges encountered in multi-agent
systems (MAS) operations, such as consensus,
collision prevention, and obstacle avoidance.
However, in order to harvest the benefits from
flocking, a proper distributed flocking control scheme
is needed. According to Reynolds's pioneering work
[1], a suitable flocking control needs to maintain
three critical rules, i.e. cohesion, separation, and
alignment. In [2]-[3], the authors represented a
theoretical framework for the design and analysis
of distributed flocking algorithms. Recently, [3]
developed a flocking control with obstacle avoidance
for MAS. Moreover, [4] derived a distributed eventtriggered hybrid flocking control for MAS.
Most of the existing control schemes in [2]-[5]
focus on maintaining the stability of each one of the
three critical flocking rules. However, if compared
with stability, the criterion of optimality is much
more preferred. In [6]-[7] the authors proposed an
optimal control for MAS to simultaneously achieve
consensus while avoiding obstacles. Unfortunately,
the methodologies in [6]-[7] cannot be used to
solve the optimal flocking problem when considering
realistic networked MAS which are commonly
affected by unknown network imperfections (e.g.
network-induced delays and packet dropouts)
and disturbances due to three drawbacks, i.e.,
i) the effects from network imperfections and
disturbance need to be carefully considered, ii) [6][7] considered simple linear system as the model
of the MAS, whereas most of the real-time MAS
are nonlinear, and iii) the practical networked MAS
dynamics are commonly unknown beforehand
due to the real-time uncertainties. To address
these deficiencies, in this article we propose a
novel distributed stochastic optimal flocking control
scheme for practical networked MAS with uncertain
dynamics, unknown network imperfections, and
disturbances.

Table of Contents for the Digital Edition of The Bridge - Issue 2, 2018