IEEE Computational Intelligence Magazine - August 2021 - 29

for every block is abandoned. This means that the operations
that are given more opportunities but result in poor performance
are removed. In order to improve search efficiency, we
eliminate the operation with the maximum w for every block
simultaneously. With this strategy, the search space which has v
blocks is significantly reduced from ||iv
and the reduced space becomes
X to (| |) ,1iv
X -
XX! - {( )}.
k
ii argmaxwo ,
kn
i
(10)
The reduction procedure is carried out repeatedly until the
optimal structure is obtained when only one operation is left in
each block.
IV. Experiments
In this section, we describe experiments on the ImageNet and
CIFAR10 datasets and compare FaUNAE with other NAS
methods and human-designed networks. In addition, we
describe experiments on the PASCAL VOC and COCO datasets.
The evolved architecture on ImageNet is applied as the
backbone of object detection.
A. Datasets
1) CIFAR-10
CIFAR-10 (Fig. 3) is a dataset containing 60,000 images.
Fifty thousand pictures are used for the training set, and the
remaining ten thousand are used for the test set. Each photo
is a 32 × 32 color photo, each pixel includes three values of
RGB, and the value range is 0-255. All photos belong to 10
different categories.
2) ImageNet
ImageNet is currently the world's largest image recognition
database. It is an ongoing research effort aimed at providing an
easy-to-access image database for researchers around the world.
At present, there are tens of millions of images in ImageNet,
and more than 20000 categories.
3) PASCAL VOC
VOC2007 contains 9963 annotated pictures and 20 categories
split into training, validation, and testing data with 24,640
objects marked. There are 11530 images in VOC2012. For the
detection task, the trainval of VOC2012 has 11540 images and
a total of 27450 objects.
4) MS COCO
The coco dataset is widely used in areas such as object detection
and instance segmentation. COCO2017 contains 118,287
training images, 5000 validation images, and 40670 testing,
with more than 80 object categories.
B. Evolution and Training Protocol
The evolution and training protocol used in our experiments
are described in this section. We first set a global average pooling
and a 2-layer MLP [12] head (hidden layer 2048-d, with
ReLU), which has a fixed-dimensional output (128-d [40])
after the hypernet and searched network. The temperature x in
6 is set to 0.2 [40], and the smoothing coefficient hyperparameter
m in (7) is set as 0.999. We use the same data enhancement
settings as MoCoV2 [11]
Experiments first evolve the initial structure 0a on an
over-parameterized network, which uses ResNet50 as the
backbone to build the architecture space (details can be found
skip_connect
N_{-1}
max_pool_3×3
conv_3×3
N_{0}
N_1
skip_connect
(a)
conv_3×3
N_{-1}
avg_pool_3×3
conv_3×3
N_{0}
N_1
avg_pool_3×3
conv_5×5
skip_connect
(b)
N_2
conv_5×5
N_3
Output
skip_connect
N_4
conv_3×3
sep_conv_5×5
skip_connect
N_3
Output
N_2
sep_conv_3×3
N_4
FIGURE 4 Detailed structure of the best cells discovered on CIFAR-10. (a) normal cells found on CIFAR10. (b) reduction cells found on CIFAR10.
AUGUST 2021 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE 29
IEEE Computational Intelligence Magazine - August 2021

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - August 2021
