Automatic Digital Modulation Classification Using STFT Spectrograms, Residual Networks, and PSO-Based Hyperparameter Optimization

Zeinali, Mansoor; Ahmad Hadi, Mohammad

doi:10.22044/jadm.2026.16887.2822

Articles in Press

Document Type : Methodologies

Authors

mansoor zeinali ¹^{, 2}
Mohammad Ahmad Hadi ¹^{, 2}

¹ Department of electrical engineering, Na.C., Islamic Azad university, Najafabad, Iran.

² Digital Processing and Machine Vision Research Center, Na.C., Islamic Azad university, Najafabad, Iran.

10.22044/jadm.2026.16887.2822

Abstract

This paper proposes an automatic modulation classification (AMC) framework that combines STFT spectrograms, a custom four-block ResNet, and Particle Swarm Optimization (PSO) for hyperparameter tuning. Its main contribution is the end-to-end integration of meta-heuristic optimization, systematic ablation analysis, and explicit evaluation under both AWGN and Rayleigh fading channels, which has not been jointly addressed in earlier STFT-ResNet studies.
The framework classifies six digital modulation schemes: ASK, PSK, FSK, QASK, QPSK, and QFSK. PSO was chosen instead of grid search and Bayesian optimization because it can efficiently handle the mixed discrete-continuous search space, including learning rate and batch size, without requiring gradient information. It also achieved convergence within only ten iterations.
A balanced synthetic dataset of 6,000 samples was generated, with 1,000 samples per modulation class and 1,024 time-domain points per sample at a sampling frequency of 100 kHz. Although the dataset is synthetic, the authors acknowledge that validating the approach on over-the-air datasets such as RadioML 2016/2018 remains important future work.
Five-fold cross-validation and ablation experiments show that the STFT representation improves accuracy by 9.6 percentage points, while PSO provides an additional statistically significant gain of 3 percentage points over a spectrogram-only baseline. The proposed model achieves 98.3% accuracy at 0 dB SNR in an AWGN channel across six tested SNR levels. Under Rayleigh fading, it reaches an unweighted average accuracy of 91.5%, improving from 81.2% at 0 dB to 97.5% at 25 dB, demonstrating strong robustness in noisy and fading conditions.

Keywords

Main Subjects

A.5. I/O and Data Communications

References

[1] X. Wang, Y. Zhao, and Z. Huang, "A Survey of Deep Transfer Learning in Automatic Modulation Classification," IEEE Transactions on Cognitive Communications and Networking, vol. 11, no. 3, 2025. doi: 10.1109/TCCN.2025.3350001.

[2] S. Ansari, K. A. Alnajjar, S. Abdallah, and M. Saad, "Automatic digital modulation recognition based on machine learning algorithms," in Proceedings of the International Conference on Communications, Computing, Cybersecurity, and Informatics (CCCI), Sharjah, UAE, 2020, pp. 1–6. doi: 10.1109/CCCI49893.2020.9256692.

[3] H. Ouamna, A. Kharbouche, Z. Madini, and Y. Zouine, "Deep Learning-Assisted Automatic Modulation Classification Using Spectrograms," Engineering, Technology and Applied Science Research, vol. 15, no. 1, pp. 19925–19932, 2025. doi: 10.48084/etasr.9036.

[4] J. C. Clement, K. Sruthi, A. Arun, E. V. Krishnamurthy, M. Vasudevan, and S. Vidhya, "Deep learning based modulation classification for 5G and beyond wireless networks," Peer-to-Peer Networking and Applications, vol. 14, no. 1, pp. 319–332, 2021. doi: 10.1007/s12083-020-00944-x.

[5] Z. Zhang, H. Wang, C. Xu, Z. Feng, and P. Zhang, "Automatic modulation classification using CNN-LSTM based dual-stream structure," IEEE Transactions on Vehicular Technology, vol. 69, no. 11, pp. 13521–13531, 2020. doi: 10.1109/TVT.2020.3019659.

[6] S. Hanna, C. Dick, and D. Cabric, "Signal processing-based deep learning for blind symbol rate estimation and modulation classification," IEEE Journal on Selected Areas in Communications, vol. 40, no. 1, pp. 82–96, 2021. doi: 10.1109/JSAC.2021.3126130.

[7] S. Ansari, H. A. Khan, K. A. Alnajjar, and S. Abdallah, "Automatic modulation recognition based on genetic-algorithm-optimized machine learning models," IEEE Access, vol. 10, pp. 50265–50277, 2022. doi: 10.1109/ACCESS.2022.3173928.

[8] P. Suman and Y. A. Qu, "LightAMC: Lightweight automatic modulation classification using a dual-path deep residual shrinkage network," Artificial Intelligence, vol. 6, no. 1, Art. no. 17, 2025. doi: 10.3390/ai6010017.

[9] N. K. Pathak and V. Bajaj, "Automatic modulation classification using bimodal parallel multichannel deep learning for MIMO systems," Physical Communication, vol. 59, Art. no. 102071, 2023. doi: 10.1016/j.phycom.2023.102071.

[10] N. El-Haryqy, M. Oukessou, A. Ait Ouahman, and R. Saadane, "Improved automatic modulation classification using deep learning with additive attention mechanism," Results in Engineering, vol. 25, Art. no. 103883, 2025. doi: 10.1016/j.rineng.2025.103883.

[11] M. Z. Nisar, M. A. Khan, Z. Ahmad, J. Choi, and B. H. Kim, "A lightweight deep learning model for automatic modulation classification using residual learning and squeeze-and-excitation blocks," Applied Sciences, vol. 13, no. 8, Art. no. 5145, 2023. doi: 10.3390/app13085145.

[12] J. L. Semmlow, Biosignal and Medical Image Processing, 2nd ed. Boca Raton, FL, USA: CRC Press, 2004.

[13] B. Koonce, "ResNet 50," in Convolutional Neural Networks with Swift for TensorFlow: Image Recognition and Dataset Categorization, Berkeley, CA, USA: Apress, 2021, pp. 63–72. doi: 10.1007/978-1-4842-6168-2_6.

[14] S. Targ, D. Almeida, and K. Lyman, "Resnet in resnet: Generalizing residual architectures," arXiv preprint arXiv:1603.08029, 2016.

[15] F. He, T. Liu, and D. Tao, "Why ResNet works? Residuals generalize," IEEE Transactions on Neural Networks and Learning Systems, vol. 31, no. 12, pp. 5349–5362, 2020. doi: 10.1109/TNNLS.2020.2966319.

[16] C. Zhang, S. Bengio, M. Hardt, and B. Recht, "Understanding deep learning requires rethinking generalization," in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2021, pp. 3550–3559.

[17] S. Hayou, J.-F. Ton, A. Doucet, and Y. W. Teh, "Stable ResNet," in Proceedings of the 24th International Conference on Artificial Intelligence and Statistics (AISTATS), 2021, pp. 1324–1332.

[18] S. Asadi Amiri and Z. Davoudi, "Enhanced Deep Learning Approaches for Wildfire Detection Using Satellite Imagery," Journal of AI and Data Mining, vol. 13, no. 4, pp. 491-500, 2025 doi: 10.22044/jadm.2025.15837.2699.

[19] T. Fawcett, "An introduction to ROC analysis," Pattern Recognition Letters, vol. 27, no. 8, pp. 861–874, 2006. doi: 10.1016/j.patrec.2005.10.010.

[20] Z. Elkhatib, A. Al-Fuqaha, B. Brik, A. Ksentini, and M. Guizani, "Radio modulation classification using combinatorial deep learning," IEEE Access, vol. 12, pp. 17552–17570, 2024. doi: 10.1109/ACCESS.2024.3357500.

[21] S. Mohsen, A. M. Ali, and A. Emam, "Automatic modulation recognition using CNN deep learning models," Multimedia Tools and Applications, vol. 83, no. 3, pp. 7035–7056, 2024. doi: 10.1007/s11042-023-15521-0.

Automatic Digital Modulation Classification Using STFT Spectrograms, Residual Networks, and PSO-Based Hyperparameter Optimization

References

References

Articles in Press, Accepted Manuscript Available Online from 18 July 2026

Articles in Press, Accepted Manuscript
Available Online from 18 July 2026