IEEE Consumer Electronics Magazine - March 2018 - 59

size increased for each layer, starting with 3 × 3 for the first
layer and 11 × 11 for the output layer. We did not use any
pooling in the network, and again we performed the batch
normalization procedure after each convolutional layer.
We trained these three networks using the Nesterov momentum method for the binary cross-entropy loss function on the
expanded CASIA 1000 data set.

-▼ The second property is the distance of this layer from
the input layer. For example, (9C,4) is the label of the
node with a 9 × 9 convolutional operation and having
distance 4 from the input data node.
3) We placed these graphs in parallel, sharing the same input
and output (see Figure 4). We assigned the same label to
all nodes with identical properties. In this simple demonstration, all of the nodes with, e.g., the property (7C,3)
were assigned label C.
4) We applied the graph contraction operation to this graph.
In the graph contraction step, we merged all of the nodes

▼

MERGING THE NETWORKS

7×7
6ch

7×7
6ch

7×7
6ch

7×7
6ch

Output
3×3

3×3
17ch

3×3
17ch

3×3
17ch

9×9
10ch

5×5
5ch
3×3
17ch

3×3
5ch

Input

Concat

7×7
6ch

7×7
6ch

7×7
6ch

The main purpose of the semiparallel DNN (SPDNN) model
is to mix and merge several deep architectures and produce
a final model that takes advantage of
the specialized layers of each architecture but is significantly smaller than
A
B
C
D
E
F
G
the combined sizes of these networks.
(7C,1) (7C,2)
(7C,3)
(7C,3)
(7C,4)
(7C,5) (7C,6)
The approach we adopted is based on
graph optimization, and we proceeded
H
I
J
K
L
as follows.
Input
Output
(3C,1)
(3C,2) (3C,3)
(3C,4)
(3C,5)
1) We translate each network into its
H
M
C
N
corresponding graph. The graph representation for each network is shown
(3C,1)
(3C,2) (3C,3)
(3C,4)
at the bottom in Figure 3(a)-(c).
2) We assigned properties to each FIGURE 4. All of the graphs in Figure 3 are placed in parallel, sharing a single input and
output.
node.
▼ -▼ The first property is the operation of the layer and the kernel
D
E
F
G
A
B
size: C for convolutional, F for
C
fully connected layers, and P for
(7C,1) (7C,2)
(7C,3)
(7C,4)
(7C,5) (7C,6)
(7C,3)
pooling operation. (Note that, in
I
K
L
this simplified example, there
J
H
are no fully connected and pool(3C,2)
Input
Output
(3C,3)
(3C,4)
(3C,5)
ing layers; please refer to [3] for
(3C,1)
N
more complex examples.) For
M
example, 5C corresponds to a
(5C,2)
(9C,4)
convolutional layer with the 5 ×
5 kernel.
FIGURE 5. The graph contraction is applied to the graph in Figure 4.

FIGURE 6. The contracted graph is translated into the neural network. Concat: concatenation.

MARCH 2018

^

IEEE Consumer Electronics Magazine

59



Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - March 2018

Contents
IEEE Consumer Electronics Magazine - March 2018 - Cover1
IEEE Consumer Electronics Magazine - March 2018 - Cover2
IEEE Consumer Electronics Magazine - March 2018 - Contents
IEEE Consumer Electronics Magazine - March 2018 - 2
IEEE Consumer Electronics Magazine - March 2018 - 3
IEEE Consumer Electronics Magazine - March 2018 - 4
IEEE Consumer Electronics Magazine - March 2018 - 5
IEEE Consumer Electronics Magazine - March 2018 - 6
IEEE Consumer Electronics Magazine - March 2018 - 7
IEEE Consumer Electronics Magazine - March 2018 - 8
IEEE Consumer Electronics Magazine - March 2018 - 9
IEEE Consumer Electronics Magazine - March 2018 - 10
IEEE Consumer Electronics Magazine - March 2018 - 11
IEEE Consumer Electronics Magazine - March 2018 - 12
IEEE Consumer Electronics Magazine - March 2018 - 13
IEEE Consumer Electronics Magazine - March 2018 - 14
IEEE Consumer Electronics Magazine - March 2018 - 15
IEEE Consumer Electronics Magazine - March 2018 - 16
IEEE Consumer Electronics Magazine - March 2018 - 17
IEEE Consumer Electronics Magazine - March 2018 - 18
IEEE Consumer Electronics Magazine - March 2018 - 19
IEEE Consumer Electronics Magazine - March 2018 - 20
IEEE Consumer Electronics Magazine - March 2018 - 21
IEEE Consumer Electronics Magazine - March 2018 - 22
IEEE Consumer Electronics Magazine - March 2018 - 23
IEEE Consumer Electronics Magazine - March 2018 - 24
IEEE Consumer Electronics Magazine - March 2018 - 25
IEEE Consumer Electronics Magazine - March 2018 - 26
IEEE Consumer Electronics Magazine - March 2018 - 27
IEEE Consumer Electronics Magazine - March 2018 - 28
IEEE Consumer Electronics Magazine - March 2018 - 29
IEEE Consumer Electronics Magazine - March 2018 - 30
IEEE Consumer Electronics Magazine - March 2018 - 31
IEEE Consumer Electronics Magazine - March 2018 - 32
IEEE Consumer Electronics Magazine - March 2018 - 33
IEEE Consumer Electronics Magazine - March 2018 - 34
IEEE Consumer Electronics Magazine - March 2018 - 35
IEEE Consumer Electronics Magazine - March 2018 - 36
IEEE Consumer Electronics Magazine - March 2018 - 37
IEEE Consumer Electronics Magazine - March 2018 - 38
IEEE Consumer Electronics Magazine - March 2018 - 39
IEEE Consumer Electronics Magazine - March 2018 - 40
IEEE Consumer Electronics Magazine - March 2018 - 41
IEEE Consumer Electronics Magazine - March 2018 - 42
IEEE Consumer Electronics Magazine - March 2018 - 43
IEEE Consumer Electronics Magazine - March 2018 - 44
IEEE Consumer Electronics Magazine - March 2018 - 45
IEEE Consumer Electronics Magazine - March 2018 - 46
IEEE Consumer Electronics Magazine - March 2018 - 47
IEEE Consumer Electronics Magazine - March 2018 - 48
IEEE Consumer Electronics Magazine - March 2018 - 49
IEEE Consumer Electronics Magazine - March 2018 - 50
IEEE Consumer Electronics Magazine - March 2018 - 51
IEEE Consumer Electronics Magazine - March 2018 - 52
IEEE Consumer Electronics Magazine - March 2018 - 53
IEEE Consumer Electronics Magazine - March 2018 - 54
IEEE Consumer Electronics Magazine - March 2018 - 55
IEEE Consumer Electronics Magazine - March 2018 - 56
IEEE Consumer Electronics Magazine - March 2018 - 57
IEEE Consumer Electronics Magazine - March 2018 - 58
IEEE Consumer Electronics Magazine - March 2018 - 59
IEEE Consumer Electronics Magazine - March 2018 - 60
IEEE Consumer Electronics Magazine - March 2018 - 61
IEEE Consumer Electronics Magazine - March 2018 - 62
IEEE Consumer Electronics Magazine - March 2018 - 63
IEEE Consumer Electronics Magazine - March 2018 - 64
IEEE Consumer Electronics Magazine - March 2018 - 65
IEEE Consumer Electronics Magazine - March 2018 - 66
IEEE Consumer Electronics Magazine - March 2018 - 67
IEEE Consumer Electronics Magazine - March 2018 - 68
IEEE Consumer Electronics Magazine - March 2018 - 69
IEEE Consumer Electronics Magazine - March 2018 - 70
IEEE Consumer Electronics Magazine - March 2018 - 71
IEEE Consumer Electronics Magazine - March 2018 - 72
IEEE Consumer Electronics Magazine - March 2018 - 73
IEEE Consumer Electronics Magazine - March 2018 - 74
IEEE Consumer Electronics Magazine - March 2018 - 75
IEEE Consumer Electronics Magazine - March 2018 - 76
IEEE Consumer Electronics Magazine - March 2018 - 77
IEEE Consumer Electronics Magazine - March 2018 - 78
IEEE Consumer Electronics Magazine - March 2018 - 79
IEEE Consumer Electronics Magazine - March 2018 - 80
IEEE Consumer Electronics Magazine - March 2018 - 81
IEEE Consumer Electronics Magazine - March 2018 - 82
IEEE Consumer Electronics Magazine - March 2018 - 83
IEEE Consumer Electronics Magazine - March 2018 - 84
IEEE Consumer Electronics Magazine - March 2018 - 85
IEEE Consumer Electronics Magazine - March 2018 - 86
IEEE Consumer Electronics Magazine - March 2018 - 87
IEEE Consumer Electronics Magazine - March 2018 - 88
IEEE Consumer Electronics Magazine - March 2018 - 89
IEEE Consumer Electronics Magazine - March 2018 - 90
IEEE Consumer Electronics Magazine - March 2018 - 91
IEEE Consumer Electronics Magazine - March 2018 - 92
IEEE Consumer Electronics Magazine - March 2018 - 93
IEEE Consumer Electronics Magazine - March 2018 - 94
IEEE Consumer Electronics Magazine - March 2018 - 95
IEEE Consumer Electronics Magazine - March 2018 - 96
IEEE Consumer Electronics Magazine - March 2018 - 97
IEEE Consumer Electronics Magazine - March 2018 - 98
IEEE Consumer Electronics Magazine - March 2018 - 99
IEEE Consumer Electronics Magazine - March 2018 - 100
IEEE Consumer Electronics Magazine - March 2018 - 101
IEEE Consumer Electronics Magazine - March 2018 - 102
IEEE Consumer Electronics Magazine - March 2018 - 103
IEEE Consumer Electronics Magazine - March 2018 - 104
IEEE Consumer Electronics Magazine - March 2018 - 105
IEEE Consumer Electronics Magazine - March 2018 - 106
IEEE Consumer Electronics Magazine - March 2018 - 107
IEEE Consumer Electronics Magazine - March 2018 - 108
IEEE Consumer Electronics Magazine - March 2018 - 109
IEEE Consumer Electronics Magazine - March 2018 - 110
IEEE Consumer Electronics Magazine - March 2018 - 111
IEEE Consumer Electronics Magazine - March 2018 - 112
IEEE Consumer Electronics Magazine - March 2018 - 113
IEEE Consumer Electronics Magazine - March 2018 - 114
IEEE Consumer Electronics Magazine - March 2018 - 115
IEEE Consumer Electronics Magazine - March 2018 - 116
IEEE Consumer Electronics Magazine - March 2018 - 117
IEEE Consumer Electronics Magazine - March 2018 - 118
IEEE Consumer Electronics Magazine - March 2018 - 119
IEEE Consumer Electronics Magazine - March 2018 - 120
IEEE Consumer Electronics Magazine - March 2018 - Cover3
IEEE Consumer Electronics Magazine - March 2018 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20240102
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20231112
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20230910
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20230708
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20230506
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20230304
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20230102
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20221112
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20220910
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20220708
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20220506
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20220304
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20220102
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20211112
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20210910
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20210708
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20210506
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20210304
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_202010
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_202009
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_202007
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_202004
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_202003
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_202001
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201910
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201909
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201907
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201905
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201903
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201901
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201811
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201809
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201807
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201805
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201803
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_july2017
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_april2017
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_january2017
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_october2016
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_july2016
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_april2016
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_january2016
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_october2015
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_july2015
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_april2015
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_january2015
https://www.nxtbookmedia.com