Low-Complexity EP-Net–Based Signal Detection for High-Mobility MIMO-OTFS Systems Over Rayleigh and Rician Channels

Arun Kumar; Venkatachalam Revathi; Prashant Chandra Pradhan; Jayanta Kumar Baruah; Nishant Gaur; Aziz Nanthaamornphong

doi:10.29284/6s1v3w36

Authors

Arun Kumar Department of Electronics and Communication Engineering, Sikkim Manipal Institute of Technology, Sikkim Manipal University, Majitar, Rangpo, India.
Venkatachalam Revathi Department of Applied Sciences, New Horizon College of Engineering Bengaluru, India
Prashant Chandra Pradhan Department of Electronics and Communication Engineering, Sikkim Manipal Institute of Technology, Sikkim Manipal University, Majitar, Rangpo, India
Jayanta Kumar Baruah Department of Electronics and Communication Engineering, Sikkim Manipal Institute of Technology, Sikkim Manipal University, Majitar, Rangpo, India
Nishant Gaur Department of Physics, JECRC University, Jaipur, India Majitar, Rangpo, India
Aziz Nanthaamornphong College of Computing, Prince of Songkla University, Phuket, Thailand

DOI:

https://doi.org/10.29284/6s1v3w36

Keywords:

MIMO-OTFS, EP-Net detector, Rician, Rayleigh, BER

Abstract

Signal detection in delay–Doppler domain multiple-input multiple-output orthogonal time frequency space (MIMO-OTFS) systems remains challenging due to severe inter-symbol interference, high computational complexity, and pronounced performance degradation under channel estimation errors, especially in high-mobility scenarios. To address these issues, this paper proposes a low-complexity, model-driven Expectation Propagation Network (EP-Net)–based signal detection framework for MIMO-OTFS systems operating over Rayleigh and Rician fading channels. The proposed EP-Net unfolds the classical expectation propagation algorithm into a finite-depth neural network with reduced iterations, shared learnable parameters, and avoidance of repeated matrix inversions, thereby significantly lowering computational complexity while preserving probabilistic interpretability. Extensive MATLAB simulations are conducted to evaluate both bit error rate (BER) and power spectral density (PSD) performance under practical channel estimation errors. Numerical results demonstrate that, at a target BER of , the proposed EP-Net achieves 8–12 dB signal-to-noise ratio gain over conventional linear, iterative, and learning-based detectors, depending on the fading environment and estimation accuracy. Moreover, PSD analysis shows that EP-Net provides up to 50 dB reduction in out-of-band emissions, indicating superior spectral confinement and interference suppression. These results confirm that the proposed EP-Net detector offers a robust, spectrally efficient, and computationally viable solution for high-mobility and 6G-oriented MIMO-OTFS wireless communication systems.

References

[1]. L. Gaudio, G. Colavolpe and G. Caire, "OTFS vs. OFDM in the Presence of Sparsity: A Fair Comparison," in IEEE Transactions on Wireless Communications, vol. 21, no. 6, pp. 4410-4423, June 2022, doi: 10.1109/TWC.2021.3129975.

[2]. Ramachandran, M.K., Surabhi, G.D. & Chockalingam, A. OTFS: A New Modulation Scheme for High-Mobility Use Cases. J Indian Inst Sci 100, 315–336 (2020). https://doi.org/10.1007/s41745-020-00167-4.

[3]. Arun Kumar, Nishant Gaur, Aziz Nanthaamornphong, "Machine learning RNNs, SVM and NN Algorithm for Massive-MIMO-OTFS 6G Waveform with Rician and Rayleigh channel", Egyptian Informatics Journal, Volume 27, 2024, https://doi.org/10.1016/j.eij.2024.100531.

[4]. Mahamat, M.S.I., Ihonock, L.E., Manga, J.S.V.T. et al. Efficient detection technique in a MIMO system using the received signal compensation algorithm. Discov Appl Sci 7, 772 (2025). https://doi.org/10.1007/s42452-025-07402-6

[5]. M. Witzke, S. Baro, F. Schreckenbach and J. Hagenauer, "Iterative detection of MIMO signals with linear detectors," Conference Record of the Thirty-Sixth Asilomar Conference on Signals, Systems and Computers, 2002., Pacific Grove, CA, USA, 2002, pp. 289-293 vol.1, doi: 10.1109/ACSSC.2002.1197193.

[6]. X. Tan et al., "Approximate Expectation Propagation Massive MIMO Detector With Weighted Neumann-Series," in IEEE Transactions on Circuits and Systems II: Express Briefs, vol. 68, no. 2, pp. 662-666, Feb. 2021, doi: 10.1109/TCSII.2020.3017381.

[7]. Huang, X., Yuan, Y., & Li, J. (2024). MIMO Signal Detection Based on IM-LSTMNet Model. Electronics, 13(16), 3153. https://doi.org/10.3390/electronics13163153

[8]. Y. Shan, F. Wang, and Y. Hao, “Orthogonal Time Frequency Space detection via low-complexity expectation propagation,” IEEE Trans. Wireless Commun., early access, pp. 1–1, 2022, doi: 10.1109/TWC.2022.3187990.

[9]. X. Pu, Z. Sun, W. Wen, Q. Chen, and S. Jin, “A low-complexity expectation propagation detector for OTFS,” IET Signal Process.,https://doi.org/10.1049/2024/3256977. 2024.

[10]. S. Li, W. Yuan, Z. Wei and J. Yuan, "Cross Domain Iterative Detection for Orthogonal Time Frequency Space Modulation," in IEEE Transactions on Wireless Communications, vol. 21, no. 4, pp. 2227-2242, April 2022, doi: 10.1109/TWC.2021.3110125.

[11]. Y. Ge, L. Liu, S. Huang, D. G. G., Y. L. Guan and Z. Ding, "Low-Complexity Memory AMP Detector for High-Mobility MIMO-OTFS SCMA Systems," 2023 IEEE International Conference on Communications Workshops (ICC Workshops), Rome, Italy, 2023, pp. 807-812, doi: 10.1109/ICCWorkshops57953.2023.10283752.

[12]. H. Chang, A. Kosasih, W. Hardjawana, X. Qu and B. Vucetic, "Untrained Neural Network based Bayesian Detector for OTFS Modulation Systems," IEEE INFOCOM 2023 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), Hoboken, NJ, USA, 2023, pp. 1-6, doi: 10.1109/INFOCOMWKSHPS57453.2023.10225901.

[13]. Wang, Y., Zhang, Z., Li, H., Zhou, T., & Cheng, Z. (2025). Signal Detection Based on Separable CNN for OTFS Communication Systems. Entropy, 27(8), 839. https://doi.org/10.3390/e27080839

[14]. Wu, Y., Zhou, M., Lin, Y., & Liao, Z. (2025). Signal Detection Method for OTFS System Based on Feature Fusion and CNN. Electronics, 14(20), 4041. https://doi.org/10.3390/electronics14204041.

[15]. P. Mamadapur and T. D. N. Sheshadri, “Performance analysis of MIMO-OTFS with advanced detection techniques for high-mobility wireless communication,” East Afr. J. Sci. Technol. Innov., vol. 1, no. 1, pp. 1–17, Nov. 2025.

[16]. Emin Akpinar, Emir Aslandogan, Burak Ahmet Ozden, Haci Ilhan and Erdogan Aydin" Deep learning-enabled signal detection for MIMO-OTFS,” https://doi.org/10.48550/arXiv.2601.13635, Jan. 2026.

[17]. Kumar, A., Gaur, N., Aly, A.A. et al. PAPR reduction of OTFS using an automatic amplitude reduction neural network with vendermonde matrix-based PTS and SLM algorithms. J Wireless Com Network 2024, 84 (2024). https://doi.org/10.1186/s13638-024-02414-z.