IEEE Consumer Electronics Magazine - January/February 2023 - 39

Table 1. MAE and MSE values between the optimal
locations and those obtained by ML-based approaches.
MAE 105.46
CNN Q-learning DQ-learning DRL
307.95
305.49
MSE 25,770 144,503
142,180
259.45
93,756
that the proposed CNN-based approach locates
UAV-BSs much closer to optimal locations compared
to other approaches.
Figure 6 presents the instantaneous coverage
of different UAV-BS placement approaches for 500
consecutive steps, where each step is 4 seconds.
The difference between the proposed approach
and other learning methods can be as many as 10
coveredMUs (e.g., at 50th and 150th steps).
CDF of the covered MUs is illustrated in
Figure 5 considering 18,000 test scenarios. The
difference between the CNN approach and the
optimal is 1 covered MU, on average, whereas,
the average difference is, approximately, 4.5 covered
MUs between RL-based approaches and the
optimal.
Performance evaluations are performed using
a 4-core Intel i7 3.6 GHz CPU with 32 GB RAM and a
Geforce GTX 650TI GPU. We adopt a similar grid
structure and RL approach proposed in Bayerlein
et al.'s work,20 whereMUs are static. Provided that
theMUs are static for a particular deployment setting,
the performance of RL-based approach is as
good as the performance of the proposed CNNbased
approach. However, when MUs are mobile,
as in the scenario considered in this article, the
number of states inflates to a prohibitively high
level. In fact, the number of states for all deployment
scenarios for a 5050 grid is impractically
high to accommodate for a resource-constrained
UAV-BS due to limited memory. Therefore, we
merge each 1010 grid into a single state to keep
thememory requirements within the levels, which
our computing platform can handle. As such, performances
of RL-based approaches are worse
than the proposed CNN-based approach, which
does not have memory requirements as high as
the RL-based approaches. On the other hand, the
DRL-based approach does not have such a
demanding memory requirement. However, the
learning time necessary for the DRL-based
approach is exceedingly high. Indeed, the duration
January/February 2023
Figure 5. CDFs of the covered MUs.
of learning for the DRL-based approach is, at least,
an order of magnitude more than the proposed
CNN-based approach, yet, the learning process
does not converge for the DRL-based approach.
Nevertheless, we limit the learning time for the
DRL-based approach up to 10 times the learning
time of the CNN-based approach to be able to perform
fair comparisons against other approaches,
which is the main reason for the worse performance
of the DRL-based approach in comparison
to the proposed CNN-based approach. Although
our results reveal that the CNN-based approach
outperforms other approaches for this particular
use case, other learning approaches, especially
DRL-based approaches, are shown to be highly
successful in other use cases.15
The labelstobeutilizedbythe proposed CNNbased
solution are generated via the offline solutions
of the optimization model we employ. In
fact, the computational complexity is moved from
the online deployment phase to the offline training
phase. However, if the environment changes
significantly over time (e.g., some buildings are
erected or demolished in the deployment area)
then the optimization model needs to be resolved
and the CNN-based model needs to be retrained
to avoid performance degradation. Although RLbased
approaches do not require retraining, they
also need to rerun once the environment changes
significantly. Nevertheless, unlike terrestrial BSs,
UAV-BSs are not envisioned to be deployed for relatively
long periods of time (e.g., years) over the
samedeployment area,1 therefore, the occurrence
of significant changes inmost deployment areas in
39
IEEE Consumer Electronics Magazine - January/February 2023

Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - January/February 2023
