IEEE Systems, Man and Cybernetics Magazine - January 2021 - 31

100
90
80

)
(2
0

)

10

(1
8
9

×

9

×

8

(1
6

)
8

×

7

(1
4

)
7

×

6
6

×
5

×

5

(1
2

)
(1
0

(8
)
4
4

Figure 3. The classification accuracy for equal numbers of feature

subspace basis vectors for CSA and PCA for the MNIST digits and
Fashion-MNIST image sets. In CSA, the feature matrix Yk size shown is
q # p for the case where p = q, with the p + q, basis vectors in the
left (U ) and right (V ) subspaces together. For PCA, the feature vector
Yk has r components (where r = p + q for each interval on the
horizontal axis).

Broader Perspective:
Tensor-Based Information
It is apparent, from the diversity of only the small sampling
of applications we have mentioned for the matrix-based
2DPCA, B2DPCA, and CSA, that these methods have
enhanced or contributed to solutions in disciplines other
than engineering. This exemplifies the type of convergence of engineering methods with other disciplines
referred to as amplification, wherein " engineering enhances or contributes to discipline X, or vice versa " in the

Table 1. MNIST handwritten digits image
set: classification accuracy.
Accuracy (%) for a Given
Number of Coefficients*

×

3
×
3

2

×

2

(6
)

50

10

60

)

CSA (MNIST Digits)
PCA (MNIST Digits)
CSA (Fashion-MNIST)
PCA (Fashion-MNIST)

70

(4
)

Correct Match Rate (%)

row (g)-(j), and for an example FashionMNIST image in the bottom row (g)-(j),
for q # p = 25 # 25 , 18 # 18 , 10 # 10, and
6 # 6 (50, 36, 20, and 12 total basis eigenvectors, respectively). For comparison, the
top row of Figure 4(b)-(e) shows reconstruction for the example MNIST digit
image based on PCA with 50, 36, 20, and 12
eigenimages; the third row Figure 4(b)-(e)
depicts reconstruction for the example
Fashion-MNIST image based on PCA, also
for 50, 36, 20, and 12 eigenimages. Figure 4
shows that for a given number of basis vectors, reconstruction with CSA is of higher
quality. Figure 5 displays eigenvalues for
CSA and first 50 for PCA (MNIST digits).
Fashion-MNIST eigenvalue plots are similar (not shown).

Accuracy (%) for a Given
Number of Subspace Basis
Vectors†

context of the transdisciplinary systems engineering point
of view espoused in [26]. Given the growing prevalence of
information in tensor form, this should also prove true in
more and more applications for related, more general,
higher-order methods to which we have alluded, such as
MPCA, HOOI, and HOSVD. For instance, in [27], HOSVD is
used in coclustering to find coherent patterns, which are
" subsets of subsets " in several examples involving multiway (or tensor) data including genomics, financial data,

Table 2. Fashion-MNIST image set:
classification accuracy.
Accuracy (%) for a Given
Number of Coefficients*

Accuracy (%) for a Given
Number of Subspace Basis
Vectors†

Coefficients*

CSA

PCA

Basis Vectors†

CSA

PCA

Coefficients*

CSA

PCA

Basis Vectors†

CSA

PCA

4

49.59

57.90

4

49.59

57.90

4

63.01

64.89

4

63.01

64.89

9

82.48

87.65

6

82.48

77.64

9

75.91

76.09

6

75.91

70.29

16

91.94

93.22

8

91.94

85.72

16

78.19

79.52

8

78.19

73.99

25

95.15

94.91

10

95.15

88.44

25

79.93

80.81

10

79.93

75.90

36

95.85

95.51

12

95.85

90.94

36

81.49

81.55

12

81.49

78.39

49

95.58

95.47

14

95.58

93.06

49

81.97

81.79

14

81.97

78.66

64

95.42

95.11

16

95.42

93.51

64

82.42

82.06

16

82.42

79.52

81

95.23

94.94

18

95.23

94.22

81

82.74

81.76

18

82.74

79.46

100

95.03

94.81

20

95.03

94.58

100

83.05

82.02

20

83.05

80.43

*The number of coefficients for PCA is the total number of elements in the
PCA feature vector (projections onto PCA principal axes); for CSA, it is the total
number of elements in the q # p CSA feature matrix (pq coefficients).
†
The number of basis vectors is the number of retained principal components
for PCA and the total number of eigenvectors in U and V subspaces for CSA
(p + q). In this table, p = q.

	

*The number of coefficients for PCA is the total number of elements in the
PCA feature vector (projections onto PCA principal axes); for CSA, it is the total
number of elements in the q # p CSA feature matrix (pq coefficients).
†
The number of basis vectors is the number of retained principal components
for PCA and the total number of eigenvectors in U and V subspaces for CSA
(p + q). In this table, p = q.

Ja nu a r y 2021

IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE	

31
IEEE Systems, Man and Cybernetics Magazine - January 2021

Table of Contents for the Digital Edition of IEEE Systems, Man and Cybernetics Magazine - January 2021
