Helformer Transformer-Based Multimodal Framework For Early Retinal Disease Detection Using Fundus And OCT Images
DOI:
https://doi.org/10.29284/sd833351Keywords:
OCT; Fundus; CFNet; Ad Res Net; Grad-CAM; MFI-Net.Abstract
Retinal eye disease is a wide range of medical conditions that impacts the eye structure and function, resulting in visual impairments or blindness. In order to avoid these kinds of severe issues, an early eye disease detection mechanism should be developed for detecting abnormalities in the eyes. However, the images of the retina and fundus may be impacted due to the pixel smudge, noise, blur, or inadequate lighting at the particular disease portion, which reduces the accuracy of early disease diagnosis. To overcome these several limitations, a Helformer transformer-based classifier model had been proposed for detecting the eye disease earlier based on the input image. Initially, the input image is collected from both fundus and OCT for automated eye disease prediction.The fundus image are preprocessed using Weighted Wavelet Visual Perception Fusion (WWPF)for contrast enhancement and Conditional Filter Network (CFNet)for denoising the fundus image, along with that the OCT image are preprocessed using Adaptive Deep Residual Network (AdResNet) for removing the noise. Then the preprocessed image of fundus and OCT image are fed into the Gradient-weighted Class Activation Mapping (Grad-CAM) for highlighting the disease region of eye. After region detection, the fundus image are segmented using Multiscale Feature Interaction Network (MFI-Net) for segmenting the blood vessels in eye region and Semantic Graph Attention Network(SGANet) is used in OCT image for segmenting the retinal layer of the eye. To extract the features, Graph-Based Radiomics Feature Extraction Algorithm (GBRF) for fundus image and fully dense fusion neural network (FDFNN) for OCT images, which extracts the most relevant features from the image. The extracted features from both the fundus and OCT image are fused together with a fusion process. The fused feature vector can be passed to a Helformer transformer-based classifier for predicting the eye disease effectively. Finally, the output can be described as Normal, Diabetic Retinopathy, Diabetic Macular Edema and Drusen based on the input image.For eye disease classification, the proposed approachachieves a high accuracy of 98.57%, high precision of 97.05% and low error value of 1.42% respectively. These proposed model automatically learns and extracts seasonal patterns from the time-series data, which differentiate against from existing techniques.
References
[1] Rodríguez, M. A., AlMarzouqi, H., &Liatsis, P. (2022). Multi-label retinal disease classification using transformers. IEEE Journal of Biomedical and Health Informatics, 27(6), 2739–2750.
[2] Lam, B. L., Leroy, B. P., Black, G., Ong, T., Yoon, D., &Trzupek, K. (2021). Genetic testing and diagnosis of inherited retinal diseases. Orphanet Journal of Rare Diseases, 16(1), 514.
[3] Heath Jeffery, R. C., Mukhtar, S. A., McAllister, I. L., Morgan, W. H., Mackey, D. A., & Chen, F. K. (2021). Inherited retinal diseases are the most common cause of blindness in the working-age population in Australia. Ophthalmic Genetics, 42(4), 431–439.
[4] Holtes, L. K., de Bruijn, S. E., Cremers, F. P. M., &Roosing, S. (2025). Dual inheritance patterns: A spectrum of non-syndromic inherited retinal disease phenotypes with varying molecular mechanisms. Progress in Retinal and Eye Research, 104, 101308. https://doi.org/10.1016/j.preteyeres.2024.101308
[5] Roman, A. J., Cideciyan, A. V., Wu, V., Garafalo, A. V., & Jacobson, S. G. (2022). Full-field stimulus testing: Role in the clinic and as an outcome measure in clinical trials of severe childhood retinal disease. Progress in Retinal and Eye Research, 87, 101000.
[6] Gong, J., Cheung, S., Fasso-Opie, A., Galvin, O., Moniz, L. S., Earle, D., Durham, T., et al. (2021). The impact of inherited retinal diseases in the United States of America (US) and Canada from a cost-of-illness perspective. Clinical Ophthalmology, 15, 2855–2866.
[7] Tayal, A., Gupta, J., Solanki, A., Bisht, K., Nayyar, A., &Masud, M. (2022). DL-CNN-based approach with image processing techniques for diagnosis of retinal diseases. Multimedia Systems, 28(4), 1417–1438.
[8] Hemalakshmi, G. R., Murugappan, M., Sikkandar, M. Y., Begum, S. S., & Prakash, N. B. (2024). Automated retinal disease classification using hybrid transformer model (SViT) using optical coherence tomography images. Neural Computing and Applications, 36(16), 9171–9188.
[9] Sheet, S. S. M., Tan, T.-S., As’ari, M. A., Hitam, W. H. W., &Sia, J. S. Y. (2022). Retinal disease identification using upgraded CLAHE filter and transfer convolution neural network. ICT Express, 8(1), 142–150.
[10] Yan, A. L., Du, S. W., &Palczewski, K. (2023). Genome editing, a superior therapy for inherited retinal diseases. Vision Research, 206, 108192.
[11] Upadhyay, P. K., Rastogi, S., & Kumar, K. V. (2022). Coherent convolution neural network based retinal disease detection using optical coherence tomographic images. Journal of King Saud University - Computer and Information Sciences, 34(10), 9688–9695.
[12] Schofield, D., Kraindler, J., Tan, O., Shrestha, R., Jelovic, D., West, S., Ma, A., Grigg, J., & Jamieson, R. V. (2022). Patient-reported health-related quality of life in individuals with inherited retinal diseases. Ophthalmology Science, 2(1), 100106.
[13] Kayadibi, İ., &Güraksın, G. E. (2023). An explainable fully dense fusion neural network with deep support vector machine for retinal disease determination. International Journal of Computational Intelligence Systems, 16(1), 28.
[14] Chowa, S. S., Bhuiyan, M. R. I., Payel, I. J., Karim, A., Khan, I. U., Montaha, S., Hasan, M. Z., Jonkman, M., &Azam, S. (2025). A low complexity efficient deep learning model for automated retinal disease diagnosis. Journal of Healthcare Informatics Research, 9(1), 1–40.
[15] Woof, W. A., de Guimarães, T. A. C., Al-Khuzaei, S., Daich Varela, M., Sen, S., Bagga, P., Mendes, B., et al. (2025). Quantification of fundus autofluorescence features in a molecularly characterized cohort of >3500 patients with inherited retinal disease from the United Kingdom. Ophthalmology Science, 5(2), 100652.
[16] Menaouer, B., Dermane, Z., Kebir, N. E. H., &Matta, N. (2022). Diabetic retinopathy classification using hybrid deep learning approach. SN Computer Science, 3(5), 357.
[17] Chavan, R., & Pete, D. (2024). Automatic multi-disease classification on retinal images using multilevel glowworm swarm convolutional neural network. Journal of Engineering and Applied Science, 71(1), 26.
[18] Latif, J., Tu, S., Xiao, C., Rehman, S. U., Imran, A., &Latif, Y. (2022). ODGNet: A deep learning model for automated optic disc localization and glaucoma classification using fundus images. SN Applied Sciences, 4(4), 98.
[19] Al-Kahtani, N., Varela-Aldás, J., Aljarbouh, A., Ishak, M. K., &Mostafa, S. M. (2025). Discrete migratory bird optimizer with deep transfer learning aided multi-retinal disease detection on fundus imaging. Results in Engineering, 26, 104574.
[20] Al-Antary, M. T., &Arafa, Y. (2021). Multi-scale attention network for diabetic retinopathy classification. IEEE Access, 9, 54190–54200.
[21] Zhang, W., Zhou, L., Zhuang, P., Li, G., Pan, X., Zhao, W., & Li, C. (2023). Underwater image enhancement via weighted wavelet visual perception fusion. IEEE Transactions on Circuits and Systems for Video Technology, 34(4), 2469–2483.
[22] Zuo, Y., Yao, W., Zeng, Y., Xie, J., Fang, Y., Huang, Y., & Jiang, W. (2024). CFNet: Conditional filter learning with dynamic noise estimation for real image denoising. Knowledge-Based Systems, 284, 111320.
[23] Gopalan, K., Srinivasan, S., Singh, M., Mathivanan, S. K., &Moorthy, U. (2025). Corn leaf disease diagnosis: Enhancing accuracy with ResNet152 and Grad-CAM for explainable AI. BMC Plant Biology, 25(1), 440.
[24] Ye, Y., Pan, C., Wu, Y., Wang, S., & Xia, Y. (2022). MFI-Net: Multiscale feature interaction network for retinal vessel segmentation. IEEE Journal of Biomedical and Health Informatics, 26(9), 4551–4562.
[25] Jorreia, O., Gonçalves, N., &Cortesão, R. (2025). Graph-based radiomics feature extraction from 2D retina images. IEEE Access, 13, 125359–125373.
[26] Sheeba, M. C., & Christopher, C. S. (2025). Adaptive deep residual network for image denoising across multiple noise levels in medical, nature, and satellite images. Ain Shams Engineering Journal, 16(1), 103188.
[27] Li, P., Liu, Y., Cui, Z., Yang, F., Zhao, Y., Lian, C., &Gao, C. (2022). Semantic graph attention with explicit anatomical association modeling for tooth segmentation from CBCT images. IEEE Transactions on Medical Imaging, 41(11), 3116–3127.
[28] Kayadibi, İ., &Güraksın, G. E. (2023). An explainable fully dense fusion neural network with deep support vector machine for retinal disease determination. International Journal of Computational Intelligence Systems, 16(1), 28.
[29] Kehinde, T. O., Adedokun, O. J., Joseph, A., Kabirat, K. M., Akano, H. A., &Olanrewaju, O. A. (2025). Helformer: An attention-based deep learning model for cryptocurrency price forecasting. Journal of Big Data, 12(1), 81.
[30] Linchundan. (2019). Fundus image 1000 [Data set]. Kaggle.https://www.kaggle.com/datasets/linchundan/fundusimage1000
[31] Mooney, P. (2018). Kermany 2018 [Data set]. Kaggle. https://www.kaggle.com/datasets/paultimothymooney/kermany2018
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
This work is licensed under a Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
