IEEE Systems, Man and Cybernetics Magazine - January 2021 - 30

correct match rate) for each combination is the average
for the 10  rounds of cross validation. Results for these
combinations are shown in Figure 2(a) for the MNIST digits and Figure 2(b) for the Fashion-MNIST. Best MNIST
digits accuracy was 95.85%, for which the size of the feature matrix Yk was only 6 # 6 (36 coefficients, and 12
total basis eigenvectors, six from each subspace defined
4.067318
by U and V). Fashion-MNIST best accuracy was 83.12%,
for p = 8 , q = 10 (10 # 8 feature matrix Yk ; 80 coeffi4.067317
cients, and 18 total basis eigenvectors with 10 from the
U subspace and 8 from the V subspace).
4.067316
For comparison with PCA, we utilized the same MNIST
images for training and testing as for CSA, again using
4.067315
1
2
3
4
5
10-fold cross validation, varying the number of principal
Iteration Number
components r over a range of values from 4 through 100,
and observing the effect on classification accuracy. For
direct comparison of PCA and CSA for the MNIST digits
Figure 1. A typical reconstruction RMSE ^= MSE h
and Fashion-MNIST, Figure 3 shows plots of a subset of the
for iterations 1-5 (V matrix was initialized, with the
ith column of V as the eigenvector of scatter matrix
CSA results where the q # p feature matrix Yk is square,
SU , corresponding to the ith largest eigenvalue of
with p, q ! " 2, 3, 4, 5, 6, 7, 8, 9, 10 , , a nd where r = p + q
SU) with the MNIST digits image set. The initial V is
principal components are used in PCA (giving the same
truncated to six columns (size is 28 # 6). The number
number of basis vectors for CSA and PCA). Note that for a
of columns of U and V are truncated to six for each
given number of PCA principal components, CSA with the
iteration here.
same total number of subspace basis eigenvectors for U and V produced significantly
better accuracy, particularly for 6-16 basis
vectors for the MNIST digits, and for 6-20
100
basis vectors for the Fashion-MNIST (see
90
80
Tables 1 and 2, right side).
70
However, if viewed in terms of the num60
ber of coefficients, in these experiments the
50
advantage of CSA over PCA in terms of
Best: 95.85%
20
accuracy is relatively small above a certain
p = 6, q = 6
20
15
15
threshold number of coefficients, and
10
10
6
Number PCs (q)
Number PCs (p) 6
below that threshold PCA outperforms CSA
2 2
U Subspace
V Subspace
(Tables 1 and 2, left side). CSA still has the
(a)
advantage that each basis eigenvector for
the MNIST image sets used here is 28 # 1;
for
PCA, each eigenimage is a 784 # 1 col85
umn vector.
Correct Match Rate (%)

Correct Match Rate (%)

RMSE

portion), producing the final U and V matrices, which
define the q # p feature matrix Yk = U T X k V.
We repeated this for 144 combinations of p and q,
where p, q ! " 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 15, 20 , . Accuracy (or

80
75
70
65

20

Best: 83.12%
p = 8, q = 10
0

15

10
6
Number PCs (p)
V Subspace

2

6

2
(b)

10

15

20

N
b PC
Number
PCs ((q))
U Subspace

Figure 2. The CSA classification accuracy (or correct match rate)

in percent (vertical axis) as a function of the number of basis
eigenvectors [principal components (PCs)] defining the left subspace
U and the right subspace V, for the feature matrix Yk = U T X k V. The
number of PCs (V ) is p, (V is n # p ) and the number of PCs (U ) is q
(U is m # q ). (a) For the MNIST digits, the best accuracy (95.85%) is
for p = 6,  q = 6. (b) For the Fashion-MNIST, the best accuracy (83.12%)
is for p = 8,  q = 10.

30	

IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE Janu ar y 2021

Reconstruction
An image A k from the sample image set
can be reconstructed based on CSA
t k = UU T X k VV T + A
r,
approximately as A
r = (1/M ) R kM= 1 A k is the
where the matrix A
r is
mean of the sample images, X k = A k - A
the centered version of image A k, and U
and V are the matrices in (6). The reconstruction quality is governed by the size
q # p of feature matrix Yk (where q and p
are the number of basis vectors of the subspaces defined by the columns of U and V,
respectively). In Figure 4, image reconstruction based on CSA for an example
MNIST digit image is shown on the second



IEEE Systems, Man and Cybernetics Magazine - January 2021

Table of Contents for the Digital Edition of IEEE Systems, Man and Cybernetics Magazine - January 2021

CONTENTS
IEEE Systems, Man and Cybernetics Magazine - January 2021 - Cover1
IEEE Systems, Man and Cybernetics Magazine - January 2021 - Cover2
IEEE Systems, Man and Cybernetics Magazine - January 2021 - CONTENTS
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 2
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 3
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 4
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 5
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 6
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 7
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 8
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 9
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 10
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 11
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 12
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 13
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 14
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 15
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 16
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 17
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 18
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 19
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 20
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 21
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 22
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 23
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 24
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 25
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 26
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 27
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 28
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 29
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 30
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 31
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 32
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 33
https://www.nxtbook.com/nxtbooks/ieee/smc_202110
https://www.nxtbook.com/nxtbooks/ieee/smc_202107
https://www.nxtbook.com/nxtbooks/ieee/smc_202104
https://www.nxtbook.com/nxtbooks/ieee/smc_202101
https://www.nxtbook.com/nxtbooks/ieee/smc_202010
https://www.nxtbook.com/nxtbooks/ieee/smc_202007
https://www.nxtbook.com/nxtbooks/ieee/smc_202004
https://www.nxtbook.com/nxtbooks/ieee/smc_202001
https://www.nxtbook.com/nxtbooks/ieee/smc_201910
https://www.nxtbook.com/nxtbooks/ieee/smc_201907
https://www.nxtbook.com/nxtbooks/ieee/smc_201904
https://www.nxtbook.com/nxtbooks/ieee/smc_201901
https://www.nxtbook.com/nxtbooks/ieee/smc_201810
https://www.nxtbook.com/nxtbooks/ieee/smc_201807
https://www.nxtbook.com/nxtbooks/ieee/smc_201804
https://www.nxtbook.com/nxtbooks/ieee/smc_201801
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1017
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0717
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0417
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0117
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1016
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0716
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0416
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0116
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1015
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0715
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0415
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0115
https://www.nxtbookmedia.com