Multi-Scale Attention Residual Network For Enhanced Single Image Super-Resolution With Application To Solar EUV Imagery
DOI:
https://doi.org/10.29284/tcjh4d64Keywords:
image super-resolution, attention mechanism, multi-scale feature extraction, solar image processing, STEREO EUVI, convolutional neural network.Abstract
Single-image super-resolution (SISR) remains a key problem in computer vision and image processing. Although the methods based on deep learning have achieved incredible progress, the current practices frequently fail to efficiently represent multi-scale features and distribute computational resources to meaningful image regions. The proposed network, Multi-Scale Attention Residual Network (MSARN), is an improved super-resolution architecture that incorporates a new Multi-Scale Feature Extraction Module (MSFEM) and a Dual-Path Channel-Spatial Attention (DPCSA) mechanism. The MSFEM employs parallel convolutional branches with different kernel sizes to learn fine-grained textures and coarse structural patterns, and the DPCSA adaptively updates feature responses across the channel and spatial dimensions. We also present a Gradient-Aware Loss that preserves high-frequency edge information during the reconstruction stage. Large-scale experiments on four common benchmark datasets (Set5, Set14, BSD100, and Urban100) show that MSARN has state-of-the-art performance, improving PSNR by 0.15-0.42 dB with scaling factors x2, x3, and x4. Moreover, we confirm the practicability of MSARN on solar extreme ultraviolet (EUV) images obtained with the STEREO-A/SECCHI EUVI 171A detector, which has the MSARN average PSNR of 32.50 dB and SSIM of 0.9274, exceeding SwinIR by an additional 0.57 dB, which supports the effectiveness of MSARN in scientific image enhancement with the objectives of heliophysics.
References
1. H. Greenspan et al., “Deep learning in medical imaging,” IEEE Trans. Med. Imaging, vol. 35, no. 5, pp. 1153–1159, 2016.
2. W. Zou et al., “Video SR for surveillance,” Pattern Recognit., vol. 114, 2021.
3. D. Liebel and M. Körner, “Single-image SR for multispectral remote sensing,” ISPRS Archives, vol. XLI-B3, pp. 883–890, 2016.
4. C. Dong et al., “Compression artifacts reduction by deep CNN,” in Proc. IEEE ICCV, 2015, pp. 576–584.
5. C. Dong et al., “Image SR using deep CNNs,” IEEE Trans. PAMI, vol. 38, no. 2, pp. 295–307, 2016.
6. J. Kim et al., “Accurate image SR using very deep CNNs,” in Proc. IEEE CVPR, 2016, pp. 1646–1654.
7. B. Lim et al., “Enhanced deep residual networks for SISR,” in Proc. IEEE CVPR Workshops, 2017, pp. 1132–1140.
8. Y. Zhang et al., “Residual dense network for image SR,” in Proc. IEEE CVPR, 2018, pp. 2472–2481.
9. C. Ledig et al., “Photo-realistic SISR using a GAN,” in Proc. IEEE CVPR, 2017, pp. 105–114.
10. J. Liang et al., “SwinIR: Image restoration using Swin Transformer,” in Proc. IEEE ICCV Workshops, 2021, pp. 1833–1844.
11. [J. Hu et al., “Squeeze-and-excitation networks,” in Proc. IEEE CVPR, 2018, pp. 7132–7141.
12. Y. Zhang et al., “Image SR using very deep residual channel attention,” in Proc. ECCV, 2018, pp. 286–301.
13. S. Woo et al., “CBAM: Convolutional block attention module,” in Proc. ECCV, 2018, pp. 3–19.
14. X. Wang et al., “Non-local neural networks,” in Proc. IEEE CVPR, 2018, pp. 7794–7803.
15. C. Szegedy et al., “Going deeper with convolutions,” in Proc. IEEE CVPR, 2015, pp. 1–9.
16. J. Li et al., “Multi-scale residual network for image SR,” in Proc. ECCV, 2018, pp. 517–532.
17. N. Ahn et al., “Fast, accurate, lightweight SR with cascading residual network,” in Proc. ECCV, 2018, pp. 252–268.
18. W. Shi et al., “Real-time SR using efficient sub-pixel CNN,” in Proc. IEEE CVPR, 2016, pp. 1874–1883.
19. E. Agustsson and R. Timofte, “NTIRE 2017 SISR challenge,” in Proc. IEEE CVPR Workshops, 2017, pp. 1122–1131.
20. M. Bevilacqua et al., “Low-complexity SISR based on nonneg. neighbor embedding,” in Proc. BMVC, 2012.
21. R. Zeyde et al., “On single image scale-up using sparse-representations,” in Curves and Surfaces, Springer, 2012, pp. 711–730.
22. D. Martin et al., “A database of human segmented natural images,” in Proc. IEEE ICCV, 2001, pp. 416–423.
23. J. B. Huang et al., “SISR from transformed self-exemplars,” in Proc. IEEE CVPR, 2015, pp. 5197–5206.
24. T. Dai et al., “Second-order attention network for SISR,” in Proc. IEEE CVPR, 2019, pp. 11065–11074.
25. M. L. Kaiser et al., “The STEREO mission: An introduction,” Space Sci. Rev., vol. 136, pp. 5–16, 2008.
26. C. J. Díaz Baso and A. Asensio Ramos, “Enhancing SDO/HMI images using deep learning,” Astron. Astrophys., vol. 614, A5, 2018.
27. S. Rahman et al., “SuperRes solar images using GANs,” in Proc. IEEE Big Data, 2020, pp. 4768–4777.
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
This work is licensed under a Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
