Instrumentation & Measurement Magazine 24-2 - 125

Conf. Intelligent Computing, Instrumentation and Control Technol.
(ICICICT), vol. 1, pp. 1423-1430, 2019.
[10]	S. A. Mohammed, S. Shirmohammadi, and S. Altamimi, " A
multimodal deep learning-based distributed network latency
measurement system, " IEEE Trans. Instrum. Meas., vol. 69, no. 5,
pp. 2487-2494, 2020.
[11]	F. Lewis, D. Vrabie, and V. Syrmos, Optimal Control, 3rd ed.
Hoboken, NJ, USA: John Wiley and Sons, 2012.
[12]	M. Abouheaf, W. Gueaieb, and D. Spinello, " Online multiobjective model-independent adaptive tracking mechanism
for dynamical systems, " Robotics, vol. 8, no. 4, p. 82, Sep. 2019.

Wail Gueaieb (wgueaiebg@uottawa.ca, MIEEE'04, SMIEEE'07)
is currently a Professor in the School of Electrical Engineering
and Computer Science at the University of Ottawa, Canada.
His research interests span the fields of intelligent mechatronics, robotics, and applied computational intelligence. He has
been developing data-driven control methods for highly complex systems of uncertain dynamics. He received the bachelor
and master's degrees in computer engineering and information science from Bilkent University, Turkey in 1995 and 1997,
respectively, and the Ph.D. degree in systems design engineering from the University of Waterloo, Canada, in 2001.

[Online]. Available: http://dx.doi.org/10.3390/robotics8040082.
[13]	M. Abouheaf and W. Gueaieb, " Multi-agent reinforcement
learning approach based on reduced value function
approximations, " in Proc. IEEE Int. Symp. Robotics and Intelligent
Sensors (IRIS), pp. 111-116, Oct. 2017.
[14]	M. Abouheaf, F. Lewis, M. Mahmoud, and D. Mikulski, " Discretetime dynamic graphical games: Model-free reinforcement
learning solution, " Control Theory and Technol., vol. 13, no. 1, pp.
55-69, 2015.
[15]	R. S. Sutton and A. G. Barto, Reinforcement Learning: An
Introduction, 2nd ed., ser. Second. Cambridge, MA, USA: MIT
Press, 1998.
[16]	R. GhasemAghaei, M. A. Rahman, W. Gueaieb, and A. El Saddik,
" Ant colony-based reinforcement learning algorithm for routing
in wireless sensor networks, " in Proc. 2007 IEEE Instrum. Meas.

Rami Abielmona (frami.abielmona@larus.com) is the Vice
President of Research and Engineering at Larus Technologies
Corporation, based in Ottawa, Canada. He also serves as an
Assistant Professor Dr. Abielmona with the School of Information Technology and Engineering at the University of Ottawa,
Canada, where he received his B.A.Sc., M.A.Sc. and Ph.D. degrees in computer and electrical engineering. He joined Larus
in 2007 to lead the integration of software engineering corporate knowledge with research in Artificial Intelligence (AI),
Machine Learning (ML), sensor networking, multi-sensor
data fusion and computationally intelligent processing architectures, resulting in the innovative Larus Total::Insight™
Decision Support System as well as the Total::Perception™
Systems Simulation Engine.

Technol. Conf. (IMTC 2007), pp. 1-6, 2007.
[17]	M. Abouheaf, N. Q. Mailhot, W. Gueaieb, and D. Spinello,
" Guidance mechanism for flexible-wing aircraft using
measurement-interfaced machine-learning platform, " IEEE Trans.
Instrum. Meas., vol. 69, no. 7, pp. 4637-4648, 2020.
[18]	M. Abouheaf, W. Gueaieb, and F. Lewis, " Model-free gradientbased adaptive learning controller for an unmanned flexible wing
aircraft, " Robotics, vol. 7, no. 4, p. 66, 2018.
[19]	D. Bertsekas and J. Tsitsiklis, Neuro-Dynamic Programming, 1st ed.
Cambridge, MA, USA: Athena Scientific, 1996.
[20]	D. Prokhorov and D. Wunsch, " Adaptive critic designs, " IEEE
Trans. Neural Networks, vol. 8, no. 5, pp. 997-1007, Sep. 1997.

Mohammed Abouheaf (mabouhea@uottawa.ca) has been
with the School of Electrical Engineering and Computer Science at the University of Ottawa, Canada since 2017. His
research interests include systems and controls, machine
learning, and applied mathematics with a focus in autonomous and intelligent applications. He obtained his B.Sc. and
M.Sc. degrees in electronics and communication engineering from Mansoura University, Egypt and his Ph.D. degree in
electrical engineering from University of Texas at Arlington,
Texas, USA in 2012.
Shuzheng Qu (fqu096@uottawa.ca) is currently a master's degree student in electrical and computer engineering at the
University of Ottawa, Canada, where he received his B.A.S.
degree in electrical engineering and computing technology.
His research focuses on applying reinforcement learning for
the control of robotic systems.
April 2021	

Moufid Harb (mou_d.harbg@larus.com) is a Chief Research
Scientist with Larus Technologies, based in Ottawa, Canada.
He obtained his Ph.D. degree in artificial intelligence and
mobile robotic navigation in 2001 in a collaboration research
between Damascus University Syria and Ruhr University of
Bochum Germany. He obtained his M.Sc. degree in instrument
design and application from the University of Manchester,
England in 1994. His general research interest is in the fields
of embedded computational intelligence architectures, distributed intelligent sensor systems, and real-time and non-real
time sensor modeling for computer simulation.

Algorithm 1
Adaptive Actor-Critic Implementation
Input:
  Positive definite Qℓ and Rℓ,   { x , y}
  Initial tracking error vectors Ek and strategies uˆ k
  Convergence error threshold Δ
  Learning parameters αa and αc
 Width L of a moving time window
  Maximum number of learning iterations Nt
Output:
  Tuned neural network weights W [ c ](*) and W [ a] (*)
1: stop ← false
2: while stop = false do
3:  r ← 0
4:  W [ a] ( r1), W [ c ] ( r1) ← random values
5: actor_converged ← false

IEEE Instrumentation & Measurement Magazine	125
http://dx.doi.org/10.3390/robotics8040082
Instrumentation & Measurement Magazine 24-2

Table of Contents for the Digital Edition of Instrumentation & Measurement Magazine 24-2
