Document Type : Original/Review Paper
Authors
Electrical and Computer Engineering Faculty, Semnan University, Semnan, Iran.
Abstract
This study presents a method to reconstruct a high-resolution image using a deep convolution neural network. We propose a deep model, entitled Deep Block Super Resolution (DBSR), by fusing the output features of a deep convolutional network and a shallow convolutional network. In this way, our model benefits from high frequency and low frequency features extracted from deep and shallow networks simultaneously. We use the residual layers in our model to make repetitive layers, increase the depth of the model, and make an end-to-end model. Furthermore, we employed a deep network in up-sampling step instead of bicubic interpolation method used in most of the previous works. Since the image resolution plays an important role to obtain rich information from the medical images and helps for accurate and faster diagnosis of the ailment, we use the medical images for resolution enhancement. Our model is capable of reconstructing a high-resolution image from low-resolution one in both medical and general images. Evaluation results on TSA and TZDE datasets, including MRI images, and Set5, Set14, B100, and Urban100 datasets, including general images, demonstrate that our model outperforms state-of-the-art alternatives in both areas of medical and general super-resolution enhancement from a single input image.
Keywords
[1] Greenspan, H. (2008). Super-resolution in medical imaging. Computer Journal, vol. 52, no. 1, pp. 43-63. DOI: https://doi.org/10.1093/comjnl/bxm075
[2] Isaac, J.S. & Kulkarni, R. (2015). Super resolution techniques for medical image processing. International Conference on Technologies for Sustainable Development (ICTSD), Mumbai, India, 2015. DOI: 10.1109/ICTSD.2015.7095900.
[3] Huang, Y., Shao, L. & Frangi, A.F. (2017). Simultaneous super-resolution and cross-modality synthesis of 3d medical images using weakly-supervised joint convolutional sparse coding. IEEE International Conference on Computer Vision (CVPR), Hawaii, United States, 2017.
[4] Lin, F., Fookes, C., Chandran, V. & Sridharan, S. (2007). Super-resolved faces for improved face recognition from surveillance video. International Conference on Biometrics (ICB), Seoul, Korea, 2007.
[5] Rasti, P., Uiboupin, T., Escalera, S. & Anbarjafari, Gh. (2016). Convolutional neural network super resolution for face recognition in surveillance monitoring. International Conference on Articulated Motion and Deformable Objects (AMDO), Palma de Mallorca, Spain, 2016.
[6] Zhang, L., Zhang, H., Shen, H. & Li, P. (2010). A super-resolution reconstruction algorithm for surveillance images. Signal Processing, vol. 90, no. 3, pp. 848-859. https://doi.org/10.1016/j.sigpro.2009.09.002.
[7] Dai, D., Wang, Y., Chen, Y. & Van Gool, L. (2016). Is image super-resolution helpful for other vision tasks? IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Placid, NY, USA, 2016.
[8] Zhang, H., Liu, D. & Xiong, Z. (2018). Convolutional neural network-based video super-resolution for action recognition. 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), Xi'an, China, 2018.
[9] Haris, M., Shakhnarovich, G. & Ukita, N. (2018). Task-driven super resolution: Object detection in low-resolution images. Arxiv: 1803.11316.
[10] Sajjadi, M.S., Scholkopf, B. & Hirsch, M. (2017). Enhancement: Single image super-resolution through automated texture synthesis. IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 2017.
[11] Zhang, Y., Bai, Y., Ding, M. & Ghanem, B. (2018). Sod-mtgan: Small object detection via multi-task generative adversarial network. European Conference on Computer Vision (ECCV), Munich, Germany, 2018. DOI: https://doi.org/10.1007/978-3-030-01261-8_13.
[12] Yang, Ch. Y., Ma, Ch. & Yang, M. H. (2014). Single-Image Super-Resolution: A Benchmark, European Conference on Computer Vision (ECCV), Zurich, Switzerland, 2014. DOI: https://doi.org/10.1007/978-3-319-10593-2_25.
[13] Rastgoo, R., Kiani, K. & Escalera, S. (2020). Hand sign language recognition using multi-view hand skeleton, Expert Systems with Applications, nol. 150, no. 113336. DOI: https://doi.org/10.1016/j.eswa.2020.113336.
[14] Rastgoo, R., Kiani, K. & Escalera, S. (2018). Multi-Modal Deep Hand Sign Language Recognition in Still Images Using Restricted Boltzmann Machine. Entropy 2018, vol. 20, no. 809.
[15] Rastgoo, R., Kiani, K. & Escalera, S. (2020). Video-based isolated hand sign language recognition using a deep cascaded model, Multimedia Tools and Applications. DOI: https://doi.org/10.1007/s11042-020-09048-5.
[16] Asadolahzade Kermanshahi, M. & Homayounpour, M.M. (2019) Improving Phoneme Sequence Recognition using Phoneme Duration Information in DNN-HSMM. Journal of AI and Data Mining (JAIDM), vol. 7, no. 1, pp. 137-147.
[17] Torfi, A., Shirvani, R.A., Keneshloo, Y., Tavaf, N. & Fox, E.A. (2020). Natural Language Processing Advancements by Deep Learning: A Survey. ArXiv: 2003.01200v2.
[18] Chao Dong, Ch., Change-Loy, Ch., He, K. & Tang, X. (2014). Learning a deep convolutional network for image super-resolution. Proceedings of European Conference on Computer Vision (ECCV), Zurich, Switzerland, 2014. DOI: https://doi.org/10.1007/978-3-319-10593-2_13.
[19] Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A. & et al. (2017). Photorealistic single image super-resolution using a generative adversarial network. IEEE International Conference on Computer Vision (CVPR), Hawaii, United States, 2017. DOI: 10.1109/CVPR.2017.19.
[20] Wang, Zh., Chen, J. & Hoi, S.C.H. (2020). Deep Learning for Image Super-resolution: A Survey. arXiv:1902.06068v1.
[21] Michaeli, T. & Irani, M. (2013). Nonparametric blind super-resolution. In: IEEE International Conference on Computer Vision (CVPR), Portland, Oregon, 2013.
[22] Horé, A. & Ziou, D. (2010). Image Quality Metrics: PSNR vs. SSIM, 20th International Conference on Pattern Recognition, Istanbul, Turkey, 2010.
[23] Bevilacqua, M., Roumy, A., Guillemot, C. & Alberi-Morel, M.-L. (2012). Low-complexity single-image super-resolution based on nonnegative neighbor embedding. British Machine Vision Conference (BMVC) 2012, pp. 1–10.
[24] Zeyde, R., Elad, M. & Protter, M. (2010). On single image scale-up using sparse-representations. International Conference on Curves and Surfaces, Avignon, France, 2010. DOI: https://doi.org/10.1007/978-3-642-27413-8_47.
[25] Huang, J.B., Singh, A. & Ahuja, N. (2015). Single Image Super-Resolution from Transformed Self-Exemplars. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Massachusetts, USA, 2015.
[26] Martin, D., Fowlkes, C., Tal, D. & Malik, J. (2001). A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. Proceedings of IEEE International Conference on Computer Vision, Canada, vol. 2, 2001, pp. 416–423.
[27] Structural MRI Datasets (2020), Available: https://www.kaggle.com/ilknuricke/neurohackinginrimages.
[28] Timofte, R., Smet, V.D. & Gool, L.V. (2014). A+: Adjusted anchored neighborhood regression for fast super-resolution. Proceedings of Asian Conference on Computer Vision (ACCV), Singapore, 2014. DOI: https://doi.org/10.1007/978-3-319-16817-3_8.
[29] Schulter, S., Lesistner, C. & Bischof, H. (2015). Fast and accurate image upscaling with super-resolution forests. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, Massachusetts, USA, 2015. DOI: 10.1109/CVPR.2015.7299003.
[30] Salvador J. & Pérez-Pellitero, E. (2015). Naive Bayes super-resolution forest. Proceedings of IEEE International Conference on Computer Vision (CVPR), Santiago, Chile, 2015. DOI: 10.1109/ICCV.2015.45.
[31] Dong, Ch., Loy, Ch. Ch., He, K. & Tang, X. (2016). Image super-resolution using deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 2, pp. 295–307. DOI: 10.1109/TPAMI.2015.2439281.
[32] Gu, S., Zuo, W., Xie, Q., Meng, D., Feng, X. & Zhang, L. (2015). Convolutional sparse coding for image super-resolution. Proceedings of IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, 2015. DOI: 10.1109/ICCV.2015.212.
[33] Wang, Z., Liu, D., Yang, J., Han, W. & Huang, T. (2015). Deep networks for image super-resolution with sparse prior. Proceedings of IEEE International Conference on Computer Vision (ICCV), 2015. DOI: 10.1109/ICCV.2015.50.
[34] Wang, Y., Wang, L., Wang, H. & Li, P. (2019). End-to-End Image Super-Resolution via Deep and Shallow Convolutional Networks. IEEE Access, Vol. 7, pp. 31959 – 31970. DOI: 10.1109/ACCESS.2019.2903582.
)