Instrumentation & Measurement Magazine 25-6 - 28

Fig. 5. RTT distribution in the two datasets. (a) Cumulative distribution function; (b) Probability mass function.
Table 2 - Deme's performance result summary of the 4 approaches
Model
Deme with no changes
Deme with transfer learning using the smaller Swarmio dataset
Deme with transfer learning using the expanded Swarmio dataset
Deme trained from scratch with the expanded Swarmio dataset
overall model for the best possible performance. Depending
on the application, transfer learning may be needed frequently
to ensure meeting the expected performance continuously.
We performed 10-fold cross-validated transfer learning of
Deme using the new Swarmio dataset. However, because that
dataset only has 100s of points, we reached an accuracy of only
81%. Although a good improvement over the aforementioned
74%, the performance still did not reach the required threshold
of 90%. This indicated that we needed more data. So we
had our partner Swarmio collect RTT data for a couple of days
through their portal, leading to a dataset with tens of thousands
of data points. The RTT distribution of this expanded
Swarmio dataset was almost identical to the smaller Swarmio
dataset shown in Fig. 5.
We then used this expanded Swarmio dataset and performed
both transfer learning and training from scratch, the
latter as a point of reference to see the advantages of transfer
learning. For training from scratch, we simply trained the
CNN model of Fig. 3 from scratch with the new Swarmio data.
For transfer learning, after the initial lab training, we froze
Deme's CNN layers except for the last four layers. When the
new Swarmio data came in, instead of randomly initializing
the last four layers, we loaded the outdated weights and
trained the model with the Swarmio data. For both transfer
learning and training from scratch, we divided Swarmio's dataset
into three parts for training (60%), validation (10%), and
testing (30%). We allocated 30% of the data for testing to make
28
Average
Accuracy
74%
81%
93%
63%
NPRE
0.73
0.58
0.33
0.81
Training time
(minutes)
14
33
187
sure our model is robust and not overfitted to the training data,
especially given that in our application, the distribution of the
data in the real world can noticeably change from the training
data, as was demonstrated. For regularization, we used the
well-known Monte-Carlo dropout method.
Results and Discussion
We compared Deme's performance on the Swarmio data without
changes in Deme, with training Deme from scratch, and
with transfer learning performed on Deme. The evaluation
was done on the same machine used to train Deme in the lab,
Fig. 6. Deme's cumulative distribution of relative errors on the expanded
Swarmio dataset.
IEEE Instrumentation & Measurement Magazine
September 2022

Instrumentation & Measurement Magazine 25-6

Table of Contents for the Digital Edition of Instrumentation & Measurement Magazine 25-6

Instrumentation & Measurement Magazine 25-6 - Cover1
Instrumentation & Measurement Magazine 25-6 - Cover2
Instrumentation & Measurement Magazine 25-6 - 1
Instrumentation & Measurement Magazine 25-6 - 2
Instrumentation & Measurement Magazine 25-6 - 3
Instrumentation & Measurement Magazine 25-6 - 4
Instrumentation & Measurement Magazine 25-6 - 5
Instrumentation & Measurement Magazine 25-6 - 6
Instrumentation & Measurement Magazine 25-6 - 7
Instrumentation & Measurement Magazine 25-6 - 8
Instrumentation & Measurement Magazine 25-6 - 9
Instrumentation & Measurement Magazine 25-6 - 10
Instrumentation & Measurement Magazine 25-6 - 11
Instrumentation & Measurement Magazine 25-6 - 12
Instrumentation & Measurement Magazine 25-6 - 13
Instrumentation & Measurement Magazine 25-6 - 14
Instrumentation & Measurement Magazine 25-6 - 15
Instrumentation & Measurement Magazine 25-6 - 16
Instrumentation & Measurement Magazine 25-6 - 17
Instrumentation & Measurement Magazine 25-6 - 18
Instrumentation & Measurement Magazine 25-6 - 19
Instrumentation & Measurement Magazine 25-6 - 20
Instrumentation & Measurement Magazine 25-6 - 21
Instrumentation & Measurement Magazine 25-6 - 22
Instrumentation & Measurement Magazine 25-6 - 23
Instrumentation & Measurement Magazine 25-6 - 24
Instrumentation & Measurement Magazine 25-6 - 25
Instrumentation & Measurement Magazine 25-6 - 26
Instrumentation & Measurement Magazine 25-6 - 27
Instrumentation & Measurement Magazine 25-6 - 28
Instrumentation & Measurement Magazine 25-6 - 29
Instrumentation & Measurement Magazine 25-6 - 30
Instrumentation & Measurement Magazine 25-6 - 31
Instrumentation & Measurement Magazine 25-6 - 32
Instrumentation & Measurement Magazine 25-6 - 33
Instrumentation & Measurement Magazine 25-6 - 34
Instrumentation & Measurement Magazine 25-6 - 35
Instrumentation & Measurement Magazine 25-6 - 36
Instrumentation & Measurement Magazine 25-6 - 37
Instrumentation & Measurement Magazine 25-6 - 38
Instrumentation & Measurement Magazine 25-6 - 39
Instrumentation & Measurement Magazine 25-6 - 40
Instrumentation & Measurement Magazine 25-6 - 41
Instrumentation & Measurement Magazine 25-6 - 42
Instrumentation & Measurement Magazine 25-6 - 43
Instrumentation & Measurement Magazine 25-6 - 44
Instrumentation & Measurement Magazine 25-6 - 45
Instrumentation & Measurement Magazine 25-6 - 46
Instrumentation & Measurement Magazine 25-6 - Cover3
Instrumentation & Measurement Magazine 25-6 - Cover4
https://www.nxtbook.com/allen/iamm/26-6
https://www.nxtbook.com/allen/iamm/26-5
https://www.nxtbook.com/allen/iamm/26-4
https://www.nxtbook.com/allen/iamm/26-3
https://www.nxtbook.com/allen/iamm/26-2
https://www.nxtbook.com/allen/iamm/26-1
https://www.nxtbook.com/allen/iamm/25-9
https://www.nxtbook.com/allen/iamm/25-8
https://www.nxtbook.com/allen/iamm/25-7
https://www.nxtbook.com/allen/iamm/25-6
https://www.nxtbook.com/allen/iamm/25-5
https://www.nxtbook.com/allen/iamm/25-4
https://www.nxtbook.com/allen/iamm/25-3
https://www.nxtbook.com/allen/iamm/instrumentation-measurement-magazine-25-2
https://www.nxtbook.com/allen/iamm/25-1
https://www.nxtbook.com/allen/iamm/24-9
https://www.nxtbook.com/allen/iamm/24-7
https://www.nxtbook.com/allen/iamm/24-8
https://www.nxtbook.com/allen/iamm/24-6
https://www.nxtbook.com/allen/iamm/24-5
https://www.nxtbook.com/allen/iamm/24-4
https://www.nxtbook.com/allen/iamm/24-3
https://www.nxtbook.com/allen/iamm/24-2
https://www.nxtbook.com/allen/iamm/24-1
https://www.nxtbook.com/allen/iamm/23-9
https://www.nxtbook.com/allen/iamm/23-8
https://www.nxtbook.com/allen/iamm/23-6
https://www.nxtbook.com/allen/iamm/23-5
https://www.nxtbook.com/allen/iamm/23-2
https://www.nxtbook.com/allen/iamm/23-3
https://www.nxtbook.com/allen/iamm/23-4
https://www.nxtbookmedia.com