Instrumentation & Measurement Magazine 24-2 - 48

dictionaries. Li et al. introduced a sparse and low-rank representation model for joint medical image fusion, denoising
and enhancement [31]. They incorporated several low-rank
and sparse regularization terms into the dictionary learning
model. In the image decomposition model, they imposed a
weighted nuclear norm and sparse constraint on the sparse
component to remove noise and enhance textural details.

of the JSR model for multimodality image fusion [36]. In the
generalized JSR model, the dictionaries for the common and
innovation components are different.
In [37], Jiang and Wang presented another multi-component SR-based image fusion approach based on the MCA
model [33], which can be formulated as:

Multi-component SR-based Methods
It is worth noting that all the SR-based fusion methods mentioned above adopt single-component SR models for sparse
decomposition, namely, only one kind of sparse component is
tackled in these methods. Given that images usually contain
complicated contents and can be separated into different kinds
of components, some multi-component SR models have been
introduced for multimodality image fusion. In comparison to
the single-component SR-based methods, the multi-component SR-based approaches can handle different components
separately by taking their own characteristics into consideration, which means that the flexibility in designing fusion
strategy is much improved. Two representative examples are
joint sparse representation (JSR) [32] and morphological component analysis (MCA) [33].
The JSR model [32] assumes that signals from different
sources share a common component and each individual
source contains an innovation component. That is, the signal from the kth source yk ∈ n (k = 1,2,...,K) can be expressed as:

y k  yC  yUk  Dx C  DxUk , (3)

where yC = DxC denotes the common component of all the signals, and yUk  DxUk denotes the innovation component for the
kth individual signal. D ∈ n×m is an over-complete dictionary
U
as previously mentioned. xC and x k are the sparse coefficient
vectors of the common component and the innovation component, respectively. Using the following notations:
 y1 
 
y


y  2    nK, D

 
y 
 k

D D 0  0 


1)
 D 0 D  0    nKm( K 
,x
    


 D 0 0  D



 xC 
 U
 x1 
 xU    m( K1),
 2
  
 U
 xK 

the JSR model can be formulated as:

min
x c ,xt

1
y  D c x c  Dt x t
2

2
2

 c x c  t x t , (5)
1

where the signal y is decomposed into a cartoon component
Dcxc and a texture component Dtxt. xc and xt denote the sparse
coefficient vectors of the cartoon and texture components obtained by dictionaries Dc and Dt, respectively. λc and λt are
regularization parameters. Various approaches have been developed in the literature to solve the problem defined in (5)
[38]. Unlike the JSR model that jointly decomposes the patches
of all the source images into a common component and different innovation components, the MCA model is performed on
each source image individually to obtain a cartoon component
and a texture comment. The cartoon component mainly focuses on piecewise smooth contents like geometric structures
in large scales, while the texture component involves fine details in small scales as well as repeated or oscillating patterns.
During the fusion process, these two components can be tackled separately with different fusion strategies according to
their own characteristics, which significantly improves the potential to achieve better fusion performance.

Global SR-based Methods
In conventional SR-based image processing issues including
image fusion, the sparse coding operation is performed on a
set of overlapping image patches independently, which results
in a representation that is multi-valued for each pixel. Wohlberg pointed out that this local patch based representation is
not optimal for the entire image [39]. The accumulating-averaging strategy is typically adopted to calculate the final
intensity value for each pixel, but it tends to smooth the fine
details in the output image due to the averaging operation. To
address this problem, the concept of convolutional sparse representation (CSR) was proposed [39]. In the CSR model, the
sparse decomposition is directly performed on the entire image rather than a local patch. As shown in Fig. 4, an image Y is
modelled as the sum over a set of M convolutions between local dictionary filters {dm} and global sparse coefficient maps
{Xm}, namely, Y   d  X , where ∗ denotes the convolution operator. The image and sparse maps are given in capitals to make a
distinction with the patch-based SR. The convolutional sparse
decomposition [39] is formulated as:
M

min x
x

s.t. y  Dx

  . (4)

It is essentially the same as the traditional SR model given
in (1) and can be solved by the sparse approximation algorithms like OMP. Based on the above JSR model, some
multimodality image fusion methods have been proposed [34,
35]. In these methods, the sparse representations of innovation components from different source images are fused and
the common component is used for reconstruction, leading to
better fusion results than traditional single-component based
methods [16, 19]. Zhang et al. introduced a generalized version
48

m1

M
M
1
Y   d m  X m    X m , (6)
1
Xm 2
m 1
m 1

2

min

where we continue to use the L2-norm and L1-norm as the
matrices can be easily stacked as column vectors. Therefore, CSR can be regarded as a global version of conventional

IEEE Instrumentation & Measurement Magazine

April 2021

Instrumentation & Measurement Magazine 24-2

Table of Contents for the Digital Edition of Instrumentation & Measurement Magazine 24-2