IEEE Systems, Man and Cybernetics Magazine - January 2021 - 27

(HOOI) [11], and higher-order singular value decomposition
(HOSVD). A good overview can be found in [11].
Returning to our main focus on the matrix-based case,
below we preface the overview of CSA with a brief background on 2DPCA and B2DPCA to provide a basis for the
development of the CSA algorithm in the next section.
Background
In the remainder of this article, we consider a set of
M grayscale sample images with m rows and n columns of pixels. Let A k denote the kth sample image
(k = 1, 2, f, M ) in the form of an m # n matrix, where the
matrix entries are the pixel intensity values (typically
0-255 for 8-bit grayscale, or normalized values in [0, 1]).
We form the set of M mean-adjusted, or centered, sample
r,
images X k where the kth-centered image X k = A k - A
M
r
^
h
and A = 1/M R k = 1 A k , is the mean image.
Preprocessing may include image registration, histogram equalization, and ensuring all images have an equal
number of horizontal pixels and an equal number of vertical pixels. In the previous work considered in this section,
various nomenclatures have been used. For consistency,
we have modified the original nomenclature somewhat to
provide consistency in the background we provide below.
2DPCA
Yang et al. [3] proposed the row version of 2DPCA in 2004.
Eigendecomposition is performed on the image covariance
or scatter matrix S R defined as:
	

1
SR = M

M

|X

T
k

X k (size n # n), (1)

k=1

obtaining the n eigenvectors of S R as the columns of an
orthogonal n # n matrix we denote as V, where the ith
column (eigenvector) corresponds to the ith largest
eigenvalue vR,i of S R . In [4], the number of eigenvectors, the " top p eigenvectors, " corresponding to a specified threshold proportion i R of the total scatter is the
smallest integer p ( p # n) for which the following
expression is true:
p

	

|v

R, i

i=1
n

|v

$ i R . (2)
R, i

i=1

Retaining only these top p eigenvectors, the size of V
becomes n # p. The feature matrix Yk is then formed by
projecting the rows of each centered image X k onto the
top p eigenvectors of S R , where these eigenvectors are
the p columns of V:
	

Yk = X k V (size of Yk is m # p, p # n). (3)

In the column version of 2DPCA proposed by Zhang and
Zhou [4] in 2005, the m eigenvectors of the scatter matrix
S C are first found (forming the columns of an m # m
orthogonal matrix U), where S C is defined as:
	

	

1
SC = M

M

|X X
k

T
k

(size m # m). (4)

k=1

The eigenvector of S C corresponding to the ith largest
eigenvalue v C,i of S C is the ith column of U. These eigenvectors form a basis for a space onto which the columns of the
centered image matrices X k are projected. The dimension
of the space spanned by the m eigenvectors may be reduced
to a value q ^q # m h, using an expression similar to (2) for a
given proportion of the total scatter. The feature matrix Yk
corresponding to each centered image is then given by
	

Yk = U T X k (U is m # q, Yk is q # n, q # m). (5)

B2DPCA
While 2DPCA projects either the rows or the columns of
the centered image matrices onto a single new subspace,
B2DPCA projects both the rows and columns onto two
new subspaces simultaneously. This approach was proposed by Zhang and Zhou [4] as a method to reduce the relatively large number of feature matrix entries (or
coefficients) in the case of 2DPCA. In B2DPCA, eigendecomposition is performed on the two scatter matrices S R
and S C defined in (1) and (4), leading to the two orthogonal matrices U (m # m) and V (n # n) already defined in
the descriptions of column 2DPCA and row 2DPCA in the
previous section. Then retaining only the top p eigenvectors (columns) in V and the top q eigenvectors (columns) in
U, the B2DPCA feature vector Yk is obtained as
	

Yk = U T X k V (size of Yk is q # p, q # m and p # n). (6)

We emphasize that after dimension reduction, U and V
sizes become m # q and n # p, respectively. Clearly, this
B2DPCA feature matrix Yk has pq coefficients ( p # n
and q # m). By comparison, the feature matrix for the row
and column 2DPCA versions has mp ( p # n) coefficients
and nq (q # m) coefficients, respectively. Typically in
practical situations, pq % mp and pq % nq.
CSA
In CSA, as proposed by Xu et al. [5], the image matrix is projected onto two subspaces simultaneously just as with
B2DPCA. The resulting feature matrix Yk is expressed
exactly as in (6) for B2DPCA, where the columns of U are
orthonormal basis vectors for one of the two subspaces and
the columns of V are orthonormal basis vectors of the second subspace. However, in contrast to B2DPCA, the matrices U and V are found through an iterative process that
seeks to minimize reconstruction error for a given number
of retained basis vectors p and q (where q of the m columns
of U and p of the n columns of V are retained). We note that
the iterations converge [6]. In (6), if p = n and q = m , given
a feature matrix Yk , the original centered image X k is perfectly reconstructed by the expression X k = UYk V T . However, for p 1 n and q 1 m , we have X k . UYk V T . Letting
t k denote the centered reconstructed image (where
X
Ja nu a r y 2021

IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE	

27



IEEE Systems, Man and Cybernetics Magazine - January 2021

Table of Contents for the Digital Edition of IEEE Systems, Man and Cybernetics Magazine - January 2021

CONTENTS
IEEE Systems, Man and Cybernetics Magazine - January 2021 - Cover1
IEEE Systems, Man and Cybernetics Magazine - January 2021 - Cover2
IEEE Systems, Man and Cybernetics Magazine - January 2021 - CONTENTS
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 2
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 3
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 4
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 5
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 6
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 7
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 8
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 9
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 10
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 11
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 12
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 13
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 14
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 15
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 16
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 17
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 18
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 19
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 20
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 21
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 22
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 23
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 24
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 25
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 26
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 27
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 28
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 29
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 30
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 31
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 32
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 33
https://www.nxtbook.com/nxtbooks/ieee/smc_202110
https://www.nxtbook.com/nxtbooks/ieee/smc_202107
https://www.nxtbook.com/nxtbooks/ieee/smc_202104
https://www.nxtbook.com/nxtbooks/ieee/smc_202101
https://www.nxtbook.com/nxtbooks/ieee/smc_202010
https://www.nxtbook.com/nxtbooks/ieee/smc_202007
https://www.nxtbook.com/nxtbooks/ieee/smc_202004
https://www.nxtbook.com/nxtbooks/ieee/smc_202001
https://www.nxtbook.com/nxtbooks/ieee/smc_201910
https://www.nxtbook.com/nxtbooks/ieee/smc_201907
https://www.nxtbook.com/nxtbooks/ieee/smc_201904
https://www.nxtbook.com/nxtbooks/ieee/smc_201901
https://www.nxtbook.com/nxtbooks/ieee/smc_201810
https://www.nxtbook.com/nxtbooks/ieee/smc_201807
https://www.nxtbook.com/nxtbooks/ieee/smc_201804
https://www.nxtbook.com/nxtbooks/ieee/smc_201801
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1017
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0717
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0417
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0117
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1016
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0716
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0416
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0116
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1015
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0715
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0415
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0115
https://www.nxtbookmedia.com