Instrumentation & Measurement Magazine 24-2 - 51
conventional SR models. The above features of different categories of SR-based fusion methods are summarized in Table 1.
Challenges and Future Trends
Although great progress has been achieved in SR-based medical
image fusion in the last decade, there still remain several notable
challenges in this field. First, certain open problems in medical
image fusion such as detail preservation, intensity consistency,
color fidelity and robustness to misregistration have not been
fully solved. Fortunately, the study of either multi-component
SR-based or global SR-based fusion methods still has large room
for further improvement, especially for the fusion strategies of
the decomposed sparse coefficients. In many existing multicomponent SR-based methods [37], [46], the fusion strategy for
the sparse coefficients of different components are similar or
even the same, which limits the performance of these methods to
a considerable extent. For the methods based on global SR models [43], [45], [46], the fusion strategies designed for the sparse
coefficient maps are simply derived from the strategy used for
sparse coefficient vectors in local SR-based methods. In the future, more effective fusion strategies are expected to improve the
performance of SR-based medical image fusion methods.
Second, previous works on medical image fusion mostly
concentrate on the two-dimensional (2D) image fusion issue.
However, most medical images are created as a 3D volume that
contains many slices collected along a certain orientation. For
3D volumetric data, fusing each slice individually using 2D fusion methods ignores the correlation across adjacent slices and
tends to lose some important volumetric details. Yin recently
proposed a tensor sparse representation (TSR)-based fusion
method to merge 3D medical images [52]. The TSR model is actually a 3D extended version of the standard SR model based
on 2D image patches. The TSR-based method is demonstrated
to be able to obtain better performance than several SR-based
2D image fusion methods on 3D medical images. Future study
could focus on developing more effective 3D sparse models
and fusion strategies for 3D medical image fusion problem.
Third, different from that most existing works on medical
image fusion concentrate on developing fusion algorithms,
few works consider the specific applications of fusion methods. In addition to the primary goal to serve for physician
observations, medical image fusion technique also has great
potential in medical image analysis tasks that require multimodality images, such as disease diagnosis, lesion detection
and tumor segmentation. With the rapid advances of artificial
intelligence (AI)-based medical image related applications,
paying more attention on the specific application of medical
image fusion is of great significance in the future.
Acknowledgments
The authors thank all of the anonymous reviewers for their insightful comments and constructive suggestions. All future
correspondence should be sent to Dr. Xun Chen at xunchen@
ustc.edu.cn.
References
[1] " The Whole Brain Atlas of Harvard Medical School, " Accessed:
Nov. 2, 2015. [Online]. Available: http://www.med.harvard.edu/
AANLIB/.
[2] A. James and B. Dasarathy, " Medical image fusion: a survey of the
state of the art, " Information Fusion, vol. 19, pp. 4-19, 2014.
[3] S. Hong, S. Choi, K. Kim, S. Park, S. Kim, S. Lee, S. Lee, J. Byun,
and M. Lee, " Diagnostic performance of [18F]FDG-PET/MRI
for liver metastasis in patients with primary malignancy: a
systematic review and meta-analysis, " European Radiology, vol. 29,
vol. 7, pp. 3553-3563, 2019.
[4] M. Farhadi, S. Mahmoudian, F. Saddadi et al., " Functional brain
abnormalities localized in 55 chronic tinnitus patients: Fusion of
SPECT coincidence imaging and MRI, " J. Cerebral Blood Flow and
Metabolism, vol. 30, no. 4, pp. 864-870, 2010.
[5] J. Kraeima, B. Dorgelo, H.A. Gulbitti et al., " Multi-modality 3D
mandibular resection planning in head and neck cancer using CT
and MRI data fusion: a clinical series, " Oral Oncology, vol. 81, pp.
22-28, 2018.
[6] M. C. Valdés Hernández, K. J. Ferguson, F. M. Chappell, and J.
M. Wardlaw, " New multispectral MRI data fusion technique for
white matter lesion segmentation: method and comparison with
thresholding in FLAIR images, " European Radiology, vol. 20, no. 7,
pp. 1684-1691, 2010.
[7] Y. Yang, Y. Que, S. Huang, and P. Lin, " Multiple visual features
measurement with gradient domain guided filtering for
multisensory image fusion, " IEEE Trans. Instrum. Meas., vol. 66,
no. 4, pp. 691-703, 2017.
[8] W. Zhao and H. Lu, " Medical image fusion and denoising with
alternating sequential filter and adaptive fractional order total
variation, " IEEE Trans. Instrum. Meas., vol. 66, no. 9, pp. 22832294, 2017.
Conclusions
[9] M. Yin, X. Liu, Y. Liu, and X. Chen, " Medical image fusion
Medical image fusion attracts increasing attention in many
fields due to its capability of handling multimodality images
for various applications. In recent years, SR has emerged as
a popular image representation approach in the field of image fusion. This paper presents an updated overview of the
recent advances achieved in SR-based medical image fusion.
April 2021
Based on the characteristics of adopted SR models, fusion
methods are categorized into the conventional local and single-component ones, the multi-component ones, the global
ones, and the latest simultaneous multi-component and global
one. The major challenges and future trends are summarized
into three aspects, i.e., the development of more effective fusion strategies for multi-component or global representation
based fusion methods, the study of SR-based methods for 3D
medical image fusion issue, and the investigation of specific
applications of medical image fusion methods.
with parameter-adaptive pulse coupled neural network in
nonsubsampled shearlet transform domain, " IEEE Trans. Instrum.
Meas., vol. 68, no. 1, pp. 49-64, 2019.
[10] W. Kong, Q. Miao, and Y. Lei, " Multimodal sensor medical image
fusion based on local difference in non-subsampled domain, "
IEEE Trans. Instrum. Meas., vol. 68, no. 4, pp. 938-951, 2019.
IEEE Instrumentation & Measurement Magazine 51
http://www.med.harvard.edu/AANLIB/
http://www.med.harvard.edu/AANLIB/
Instrumentation & Measurement Magazine 24-2
Table of Contents for the Digital Edition of Instrumentation & Measurement Magazine 24-2
No label
Instrumentation & Measurement Magazine 24-2 - No label
Instrumentation & Measurement Magazine 24-2 - Cover2
Instrumentation & Measurement Magazine 24-2 - 1
Instrumentation & Measurement Magazine 24-2 - 2
Instrumentation & Measurement Magazine 24-2 - 3
Instrumentation & Measurement Magazine 24-2 - 4
Instrumentation & Measurement Magazine 24-2 - 5
Instrumentation & Measurement Magazine 24-2 - 6
Instrumentation & Measurement Magazine 24-2 - 7
Instrumentation & Measurement Magazine 24-2 - 8
Instrumentation & Measurement Magazine 24-2 - 9
Instrumentation & Measurement Magazine 24-2 - 10
Instrumentation & Measurement Magazine 24-2 - 11
Instrumentation & Measurement Magazine 24-2 - 12
Instrumentation & Measurement Magazine 24-2 - 13
Instrumentation & Measurement Magazine 24-2 - 14
Instrumentation & Measurement Magazine 24-2 - 15
Instrumentation & Measurement Magazine 24-2 - 16
Instrumentation & Measurement Magazine 24-2 - 17
Instrumentation & Measurement Magazine 24-2 - 18
Instrumentation & Measurement Magazine 24-2 - 19
Instrumentation & Measurement Magazine 24-2 - 20
Instrumentation & Measurement Magazine 24-2 - 21
Instrumentation & Measurement Magazine 24-2 - 22
Instrumentation & Measurement Magazine 24-2 - 23
Instrumentation & Measurement Magazine 24-2 - 24
Instrumentation & Measurement Magazine 24-2 - 25
Instrumentation & Measurement Magazine 24-2 - 26
Instrumentation & Measurement Magazine 24-2 - 27
Instrumentation & Measurement Magazine 24-2 - 28
Instrumentation & Measurement Magazine 24-2 - 29
Instrumentation & Measurement Magazine 24-2 - 30
Instrumentation & Measurement Magazine 24-2 - 31
Instrumentation & Measurement Magazine 24-2 - 32
Instrumentation & Measurement Magazine 24-2 - 33
Instrumentation & Measurement Magazine 24-2 - 34
Instrumentation & Measurement Magazine 24-2 - 35
Instrumentation & Measurement Magazine 24-2 - 36
Instrumentation & Measurement Magazine 24-2 - 37
Instrumentation & Measurement Magazine 24-2 - 38
Instrumentation & Measurement Magazine 24-2 - 39
Instrumentation & Measurement Magazine 24-2 - 40
Instrumentation & Measurement Magazine 24-2 - 41
Instrumentation & Measurement Magazine 24-2 - 42
Instrumentation & Measurement Magazine 24-2 - 43
Instrumentation & Measurement Magazine 24-2 - 44
Instrumentation & Measurement Magazine 24-2 - 45
Instrumentation & Measurement Magazine 24-2 - 46
Instrumentation & Measurement Magazine 24-2 - 47
Instrumentation & Measurement Magazine 24-2 - 48
Instrumentation & Measurement Magazine 24-2 - 49
Instrumentation & Measurement Magazine 24-2 - 50
Instrumentation & Measurement Magazine 24-2 - 51
Instrumentation & Measurement Magazine 24-2 - 52
Instrumentation & Measurement Magazine 24-2 - 53
Instrumentation & Measurement Magazine 24-2 - 54
Instrumentation & Measurement Magazine 24-2 - 55
Instrumentation & Measurement Magazine 24-2 - 56
Instrumentation & Measurement Magazine 24-2 - 57
Instrumentation & Measurement Magazine 24-2 - 58
Instrumentation & Measurement Magazine 24-2 - 59
Instrumentation & Measurement Magazine 24-2 - 60
Instrumentation & Measurement Magazine 24-2 - 61
Instrumentation & Measurement Magazine 24-2 - 62
Instrumentation & Measurement Magazine 24-2 - 63
Instrumentation & Measurement Magazine 24-2 - 64
Instrumentation & Measurement Magazine 24-2 - 65
Instrumentation & Measurement Magazine 24-2 - 66
Instrumentation & Measurement Magazine 24-2 - 67
Instrumentation & Measurement Magazine 24-2 - 68
Instrumentation & Measurement Magazine 24-2 - 69
Instrumentation & Measurement Magazine 24-2 - 70
Instrumentation & Measurement Magazine 24-2 - 71
Instrumentation & Measurement Magazine 24-2 - 72
Instrumentation & Measurement Magazine 24-2 - 73
Instrumentation & Measurement Magazine 24-2 - 74
Instrumentation & Measurement Magazine 24-2 - 75
Instrumentation & Measurement Magazine 24-2 - 76
Instrumentation & Measurement Magazine 24-2 - 77
Instrumentation & Measurement Magazine 24-2 - 78
Instrumentation & Measurement Magazine 24-2 - 79
Instrumentation & Measurement Magazine 24-2 - 80
Instrumentation & Measurement Magazine 24-2 - 81
Instrumentation & Measurement Magazine 24-2 - 82
Instrumentation & Measurement Magazine 24-2 - 83
Instrumentation & Measurement Magazine 24-2 - 84
Instrumentation & Measurement Magazine 24-2 - 85
Instrumentation & Measurement Magazine 24-2 - 86
Instrumentation & Measurement Magazine 24-2 - 87
Instrumentation & Measurement Magazine 24-2 - 88
Instrumentation & Measurement Magazine 24-2 - 89
Instrumentation & Measurement Magazine 24-2 - 90
Instrumentation & Measurement Magazine 24-2 - 91
Instrumentation & Measurement Magazine 24-2 - 92
Instrumentation & Measurement Magazine 24-2 - 93
Instrumentation & Measurement Magazine 24-2 - 94
Instrumentation & Measurement Magazine 24-2 - 95
Instrumentation & Measurement Magazine 24-2 - 96
Instrumentation & Measurement Magazine 24-2 - 97
Instrumentation & Measurement Magazine 24-2 - 98
Instrumentation & Measurement Magazine 24-2 - 99
Instrumentation & Measurement Magazine 24-2 - 100
Instrumentation & Measurement Magazine 24-2 - 101
Instrumentation & Measurement Magazine 24-2 - 102
Instrumentation & Measurement Magazine 24-2 - 103
Instrumentation & Measurement Magazine 24-2 - 104
Instrumentation & Measurement Magazine 24-2 - 105
Instrumentation & Measurement Magazine 24-2 - 106
Instrumentation & Measurement Magazine 24-2 - 107
Instrumentation & Measurement Magazine 24-2 - 108
Instrumentation & Measurement Magazine 24-2 - 109
Instrumentation & Measurement Magazine 24-2 - 110
Instrumentation & Measurement Magazine 24-2 - 111
Instrumentation & Measurement Magazine 24-2 - 112
Instrumentation & Measurement Magazine 24-2 - 113
Instrumentation & Measurement Magazine 24-2 - 114
Instrumentation & Measurement Magazine 24-2 - 115
Instrumentation & Measurement Magazine 24-2 - 116
Instrumentation & Measurement Magazine 24-2 - 117
Instrumentation & Measurement Magazine 24-2 - 118
Instrumentation & Measurement Magazine 24-2 - 119
Instrumentation & Measurement Magazine 24-2 - 120
Instrumentation & Measurement Magazine 24-2 - 121
Instrumentation & Measurement Magazine 24-2 - 122
Instrumentation & Measurement Magazine 24-2 - 123
Instrumentation & Measurement Magazine 24-2 - 124
Instrumentation & Measurement Magazine 24-2 - 125
Instrumentation & Measurement Magazine 24-2 - 126
Instrumentation & Measurement Magazine 24-2 - 127
Instrumentation & Measurement Magazine 24-2 - 128
Instrumentation & Measurement Magazine 24-2 - 129
Instrumentation & Measurement Magazine 24-2 - 130
Instrumentation & Measurement Magazine 24-2 - 131
Instrumentation & Measurement Magazine 24-2 - 132
Instrumentation & Measurement Magazine 24-2 - Cover3
Instrumentation & Measurement Magazine 24-2 - Cover4
https://www.nxtbook.com/allen/iamm/26-6
https://www.nxtbook.com/allen/iamm/26-5
https://www.nxtbook.com/allen/iamm/26-4
https://www.nxtbook.com/allen/iamm/26-3
https://www.nxtbook.com/allen/iamm/26-2
https://www.nxtbook.com/allen/iamm/26-1
https://www.nxtbook.com/allen/iamm/25-9
https://www.nxtbook.com/allen/iamm/25-8
https://www.nxtbook.com/allen/iamm/25-7
https://www.nxtbook.com/allen/iamm/25-6
https://www.nxtbook.com/allen/iamm/25-5
https://www.nxtbook.com/allen/iamm/25-4
https://www.nxtbook.com/allen/iamm/25-3
https://www.nxtbook.com/allen/iamm/instrumentation-measurement-magazine-25-2
https://www.nxtbook.com/allen/iamm/25-1
https://www.nxtbook.com/allen/iamm/24-9
https://www.nxtbook.com/allen/iamm/24-7
https://www.nxtbook.com/allen/iamm/24-8
https://www.nxtbook.com/allen/iamm/24-6
https://www.nxtbook.com/allen/iamm/24-5
https://www.nxtbook.com/allen/iamm/24-4
https://www.nxtbook.com/allen/iamm/24-3
https://www.nxtbook.com/allen/iamm/24-2
https://www.nxtbook.com/allen/iamm/24-1
https://www.nxtbook.com/allen/iamm/23-9
https://www.nxtbook.com/allen/iamm/23-8
https://www.nxtbook.com/allen/iamm/23-6
https://www.nxtbook.com/allen/iamm/23-5
https://www.nxtbook.com/allen/iamm/23-2
https://www.nxtbook.com/allen/iamm/23-3
https://www.nxtbook.com/allen/iamm/23-4
https://www.nxtbookmedia.com