correct match rate) for each combination is the average for the 10 rounds of cross validation. Results for these combinations are shown in Figure 2(a) for the MNIST digits and Figure 2(b) for the Fashion-MNIST. Best MNIST digits accuracy was 95.85%, for which the size of the feature matrix Yk was only 6 # 6 (36 coefficients, and 12 total basis eigenvectors, six from each subspace defined 4.067318 by U and V). Fashion-MNIST best accuracy was 83.12%, for p = 8 , q = 10 (10 # 8 feature matrix Yk ; 80 coeffi4.067317 cients, and 18 total basis eigenvectors with 10 from the U subspace and 8 from the V subspace). 4.067316 For comparison with PCA, we utilized the same MNIST images for training and testing as for CSA, again using 4.067315 1 2 3 4 5 10-fold cross validation, varying the number of principal Iteration Number components r over a range of values from 4 through 100, and observing the effect on classification accuracy. For direct comparison of PCA and CSA for the MNIST digits Figure 1. A typical reconstruction RMSE ^= MSE h and Fashion-MNIST, Figure 3 shows plots of a subset of the for iterations 1-5 (V matrix was initialized, with the ith column of V as the eigenvector of scatter matrix CSA results where the q # p feature matrix Yk is square, SU , corresponding to the ith largest eigenvalue of with p, q ! " 2, 3, 4, 5, 6, 7, 8, 9, 10 , , a nd where r = p + q SU) with the MNIST digits image set. The initial V is principal components are used in PCA (giving the same truncated to six columns (size is 28 # 6). The number number of basis vectors for CSA and PCA). Note that for a of columns of U and V are truncated to six for each given number of PCA principal components, CSA with the iteration here. same total number of subspace basis eigenvectors for U and V produced significantly better accuracy, particularly for 6-16 basis vectors for the MNIST digits, and for 6-20 100 basis vectors for the Fashion-MNIST (see 90 80 Tables 1 and 2, right side). 70 However, if viewed in terms of the num60 ber of coefficients, in these experiments the 50 advantage of CSA over PCA in terms of Best: 95.85% 20 accuracy is relatively small above a certain p = 6, q = 6 20 15 15 threshold number of coefficients, and 10 10 6 Number PCs (q) Number PCs (p) 6 below that threshold PCA outperforms CSA 2 2 U Subspace V Subspace (Tables 1 and 2, left side). CSA still has the (a) advantage that each basis eigenvector for the MNIST image sets used here is 28 # 1; for PCA, each eigenimage is a 784 # 1 col85 umn vector. Correct Match Rate (%) Correct Match Rate (%) RMSE portion), producing the final U and V matrices, which define the q # p feature matrix Yk = U T X k V. We repeated this for 144 combinations of p and q, where p, q ! " 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 15, 20 , . Accuracy (or 80 75 70 65 20 Best: 83.12% p = 8, q = 10 0 15 10 6 Number PCs (p) V Subspace 2 6 2 (b) 10 15 20 N b PC Number PCs ((q)) U Subspace Figure 2. The CSA classification accuracy (or correct match rate) in percent (vertical axis) as a function of the number of basis eigenvectors [principal components (PCs)] defining the left subspace U and the right subspace V, for the feature matrix Yk = U T X k V. The number of PCs (V ) is p, (V is n # p ) and the number of PCs (U ) is q (U is m # q ). (a) For the MNIST digits, the best accuracy (95.85%) is for p = 6, q = 6. (b) For the Fashion-MNIST, the best accuracy (83.12%) is for p = 8, q = 10. 30 IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE Janu ar y 2021 Reconstruction An image A k from the sample image set can be reconstructed based on CSA t k = UU T X k VV T + A r, approximately as A r = (1/M ) R kM= 1 A k is the where the matrix A r is mean of the sample images, X k = A k - A the centered version of image A k, and U and V are the matrices in (6). The reconstruction quality is governed by the size q # p of feature matrix Yk (where q and p are the number of basis vectors of the subspaces defined by the columns of U and V, respectively). In Figure 4, image reconstruction based on CSA for an example MNIST digit image is shown on the second

IEEE Systems, Man and Cybernetics Magazine - January 2021 - Cover1

IEEE Systems, Man and Cybernetics Magazine - January 2021 - Cover2

IEEE Systems, Man and Cybernetics Magazine - January 2021 - CONTENTS

IEEE Systems, Man and Cybernetics Magazine - January 2021 - 2

IEEE Systems, Man and Cybernetics Magazine - January 2021 - 3

IEEE Systems, Man and Cybernetics Magazine - January 2021 - 4

IEEE Systems, Man and Cybernetics Magazine - January 2021 - 5

IEEE Systems, Man and Cybernetics Magazine - January 2021 - 6

IEEE Systems, Man and Cybernetics Magazine - January 2021 - 7

IEEE Systems, Man and Cybernetics Magazine - January 2021 - 8

IEEE Systems, Man and Cybernetics Magazine - January 2021 - 9

IEEE Systems, Man and Cybernetics Magazine - January 2021 - 10

IEEE Systems, Man and Cybernetics Magazine - January 2021 - 11

IEEE Systems, Man and Cybernetics Magazine - January 2021 - 12

IEEE Systems, Man and Cybernetics Magazine - January 2021 - 13

IEEE Systems, Man and Cybernetics Magazine - January 2021 - 14

IEEE Systems, Man and Cybernetics Magazine - January 2021 - 15

IEEE Systems, Man and Cybernetics Magazine - January 2021 - 16

IEEE Systems, Man and Cybernetics Magazine - January 2021 - 17

IEEE Systems, Man and Cybernetics Magazine - January 2021 - 18

IEEE Systems, Man and Cybernetics Magazine - January 2021 - 19

IEEE Systems, Man and Cybernetics Magazine - January 2021 - 20

IEEE Systems, Man and Cybernetics Magazine - January 2021 - 21

IEEE Systems, Man and Cybernetics Magazine - January 2021 - 22

IEEE Systems, Man and Cybernetics Magazine - January 2021 - 23

IEEE Systems, Man and Cybernetics Magazine - January 2021 - 24

IEEE Systems, Man and Cybernetics Magazine - January 2021 - 25

IEEE Systems, Man and Cybernetics Magazine - January 2021 - 26

IEEE Systems, Man and Cybernetics Magazine - January 2021 - 27

IEEE Systems, Man and Cybernetics Magazine - January 2021 - 28

IEEE Systems, Man and Cybernetics Magazine - January 2021 - 29

IEEE Systems, Man and Cybernetics Magazine - January 2021 - 30

IEEE Systems, Man and Cybernetics Magazine - January 2021 - 31

IEEE Systems, Man and Cybernetics Magazine - January 2021 - 32

IEEE Systems, Man and Cybernetics Magazine - January 2021 - 33

https://www.nxtbook.com/nxtbooks/ieee/smc_202110

https://www.nxtbook.com/nxtbooks/ieee/smc_202107

https://www.nxtbook.com/nxtbooks/ieee/smc_202104

https://www.nxtbook.com/nxtbooks/ieee/smc_202101

https://www.nxtbook.com/nxtbooks/ieee/smc_202010

https://www.nxtbook.com/nxtbooks/ieee/smc_202007

https://www.nxtbook.com/nxtbooks/ieee/smc_202004

https://www.nxtbook.com/nxtbooks/ieee/smc_202001

https://www.nxtbook.com/nxtbooks/ieee/smc_201910

https://www.nxtbook.com/nxtbooks/ieee/smc_201907

https://www.nxtbook.com/nxtbooks/ieee/smc_201904

https://www.nxtbook.com/nxtbooks/ieee/smc_201901

https://www.nxtbook.com/nxtbooks/ieee/smc_201810

https://www.nxtbook.com/nxtbooks/ieee/smc_201807

https://www.nxtbook.com/nxtbooks/ieee/smc_201804

https://www.nxtbook.com/nxtbooks/ieee/smc_201801

https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1017

https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0717

https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0417

https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0117

https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1016

https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0716

https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0416

https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0116

https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1015

https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0715

https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0415

https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0115

https://www.nxtbookmedia.com