Document Type : Technical Paper
Authors
Department of Computer and Information Technology Engineering, Qazvin Branch, Islamic Azad University, Qazvin, Iran.
Abstract
In this article, we consider the problems of abnormal behavior detection in a high-crowded environment. One of the main issues in abnormal behavior detection is the complexity of the structure patterns between the frames. In this paper, social force and optical flow patterns are used to prepare the system for training the complexity of the structural patterns. The cycle GAN system has been used to train behavioral patterns. Two models of normal and abnormal behavioral patterns are used to evaluate the accuracy of the system detection. In the case of abnormal patterns used for training, due to the lack of this type of behavioral pattern, which is another challenge in detecting the abnormal behaviors, the geometric techniques are used to augment the patterns. If the normal behavioral patterns are used for training, there is no need to augment the patterns because the normal patterns are sufficient. Then, by using the cycle generative adversarial nets (cycle GAN), the normal and abnormal behaviors training will be considered separately. This system produces the social force and optical flow pattern for normal and abnormal behaviors on the first and second sides. We use the cycle GAN system both to train behavioral patterns and to assess the accuracy of abnormal behaviors detection. In the testing phase, if normal behavioral patterns are used for training, the cycle GAN system should not be able to reconstruct the abnormal behavioral patterns with high accuracy.
Keywords
[1] M. Ravanbakhsh, E. Sangineto, M. Nabi, and N. Sebe, "Training Adversarial Discriminators for Cross-channel Abnormal Event Detection in Crowds," 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1896-1904, 2019.
[2] R. Chaker, Z. A. Aghbari, and I. N. Junejo, "Social network model for crowd anomaly detection and localization," Pattern Recognition, Vol. 61, pp. 266-281, 2017.
[3] A. Samdurkar, Sh. Kamble, Ni. Thakur, and A. Patharkar, "Overview of Object Detection and Tracking based on Block Matching Techniques," Second International Conference on Research in Intelligent and Computing in Engineering, pp. 313-319, 2017.
[4] Li. Liu, W. Ouyang, X. Wang, P. Fieguth, J. Chen, X.Liu, and M. Pietikäinen, "Deep Learning for Generic Object Detection: A Survey," International Journal of Computer Vision, Vol. 128, No. 2, pp. 261-318, 2020.
[5] T. A. Mostafa, J. Uddin, and M. H. Ali, "Abnormal event detection in crowded scenarios," 2017 3rd International Conference on Electrical Information and Communication Technology (EICT), pp. 1-6, 2017.
[6] M. Bertini, A. D. Bimbo and L. Seidenari, "Multi-scale and real-time non-parametric approach for anomaly detection and localization," Computer Vision and Image Understanding, Vol. 116, No. 3, pp. 320-329, 2012.
[7] C. Lu, J. Shi, W. Wang, and J. Jia, "Fast Abnormal Event Detection," International Journal of Computer Vision, Vol. 127, No. 8, pp. 993-1011, 2019.
[8] J. Wang and Z. Xu, "Spatio-temporal texture modelling for real-time crowd anomaly detection," Computer Vision and Image Understanding, Vol. 144, pp. 177-187, 2016.
[9] Y. S. Chong and Y. H. Tay, "Abnormal Event Detection in Videos using Spatiotemporal Autoencoder," Computer Vision and Pattern Recognition, pp. 189-196, 2017.
[10] M. Ravanbakhsh, M. Nabi, E. Sangineto, L. Marcenaro, C. Regazzoni, and N. Sebe, "Abnormal event detection in videos using generative adversarial nets," 2017 IEEE International Conference on Image Processing (ICIP), pp. 1577-1581, 2017.
[11] J. Zhu, T. Park, P. Isola, and A. A. Efros, "Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks," 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2242-2251, 2017.
[12] K. E. Ko and K. B. Sim, "Deep convolutional framework for abnormal behavior detection in a smart surveillance system," Engineering Applications of Artificial Intelligence, Vol. 67, pp. 226-234, 2018.
[13] M. Paul, S. M. E. Haque, and S. Chakraborty, "Human detection in surveillance videos and its applications-a review," EURASIP Journal on Advances in Signal Processing, Vol. 2013, No. 1, pp. 176, 2018.
[14] C. Hemalatha, S. Muruganand, and R. Maheswaran, "A Survey on Real Time Object Detection, Tracking and Recognition in Image Processing," International Journal of Computer Applications, Vol. 91, No. 16, pp. 38-42, 2014.
[15] T. Anbu, M. M. Joe, and G. Murugeswari, "A comprehensive survey of detecting tampered images and localization of the tampered region," Multi-media Tools and Applications, Vol. 80, No. 2, pp. 2713-2751, 2021.
[16] Y. Xiao, Zh. Tian, J. Yu, Y. Zhang, Sh. Liu, Sh. Du and X. Lan, "A review of object detection based on deep learning," Multimedia Tools and Applications, Vol. 79, No. 33, pp. 23729-23791, 2020.
[17] A. E. Gunduz, C. Ongun, T. T. Temizel, and A. Temizel, "Density aware anomaly detection in crowded scenes," IET Computer Vision, Vol. 10, No. 5, pp. 374-381, 2016.
[18] D. Shehab and H. Ammar, "Statistical detection of a panic behavior in crowded scenes," Machine Vision and Applications, Vol. 30, No. 5, pp. 919-931, 2019.
[19] S. Ezatzadeh and M. R. Keyvanpour, "ViFa: an analytical framework for vision-based fall detection in a surveillance environment," Multi-media Tools and Applications, Vol. 78, No. 18, pp. 25515-25537, 2019.
[20] C. Vishnu, D. Singh, C. K. Mohan, and S. Babu, "Detection of motorcyclists without helmet in videos using convolutional neural network," 2017 International Joint Conference on Neural Networks (IJCNN), pp. 3036-3041, 2017.
[21] J. Wang and Z. Xu, "Crowd anomaly detection for automated video surveillance," 6th International Conference on Imaging for Crime Prevention and Detection (ICDP-15), pp. 1-6, 2015.
[22] D. Dawei, Q. Honggang, H. Qingming, Z. Wei, and Z. Changhua, "Abnormal event detection in crowded scenes based on Structural Multi-scale Motion Interrelated Patterns," 2013 IEEE International Conference on Multimedia and Expo (ICME), pp. 1-6, 2013.
[23] H. Chebi and D. Acheli, "Dynamic detection of anomalies in crowd's behavior analysis," 2015 4th International Conference on Electrical Engineering (ICEE), pp. 1-5, 2015.
[24] A. Li, Z. Miao, Y. Cen, and Q. Liang, "Abnormal event detection based on sparse reconstruction in crowded scenes," 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1786-1790, 2016.
[25] K. Vignesh, G. Yadav, and A. Sethi, "Abnormal Event Detection on BMTT-PETS 2017 Surveillance Challenge," 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 2161-2168, 2017.
[26] T. Wang, J. Chen, and H. Snoussi, "Online Detection of Abnormal Events in Video Streams," Journal of Electrical and Computer Engineering, Vol. 2013, pp. 1-12, 2013.
[27] X. Li, Y. She, D. Luo, and Zh. Yu, "A Traffic State Detection Tool for Freeway Video Surveillance System," Procedia-Social and Behavioral Sciences, Vol. 96, pp. 2453-2461, 2013.
[28] M. Manfredi, R. Vezzani, S. Calderara, and R. Cucchiara, "Detection of static groups and crowds gathered in open spaces by texture classification," Pattern Recognition Letters, Vol. 44, pp. 39-48, 2014.
[29] M. Sabokrou, M. Fayyaz, M. Fathy, Z. Moayed, and R. Klette, "Deep-anomaly: Fully convolutional neural network for fast anomaly detection in crowded scenes," Computer Vision and Image Understanding, Vol. 172, pp. 88-97, 2018.
[30] X. Zong, Y. Chen, A. Liu, R. Li, S. Liu, H. Yu, and M. Tan, "Abnormal Event Detection in Video based on Sparse Representation," 2020 15th International Conference on Computer Science and Education (ICCSE), pp. 649-653, 2020.
[31] C. Spampinato, S. Palazzo, P. D'Oro, D. Giordano, and M. Shah, "Adversarial Framework for Unsupervised Learning of Motion Dynamics in Videos," International Journal of Computer Vision, Vol. 128, No. 5, pp. 1378-1397, 2020.
[32] S. Hamdi, S. Bouindour, K. Loukil, H. Snoussi, and M. Abid, "Two-streams Fully Convolutional Networks for Abnormal Event Detection in Videos," 12th International Conference on Agents and Artificial Intelligence, pp. 514-521, 2020.
[33] I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, Sh. Ozair, A. Courville, and Y. Bengio, "Generative adversarial nets," Proceedings of the 27th International Conference on Neural Information Processing Systems, Vol. 2, pp. 2672–2680, 2014.
[34] P. Isola, J. Zhu, T. Zhou, and A. A. Efros, "Image-to-Image Translation with Conditional Adversarial Networks," 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5967-5976, 2017.
[35] E. Pejhan and M. Ghasemzadeh, "Multi-Sentence Hierarchical Generative Adversarial Network GAN (MSH-GAN) for Automatic Text-to-Image Generation," Journal of AI and Data Mining, Vol. 9, No. 4, pp. 475-485, 2021.
[36] Th. Brox, A. Bruhn, N. Papenberg, and J. Weickert, "High Accuracy Optical Flow Estimation based on a Theory for Warping," In: Pajdla T.. Matas J. (eds) Computer Vision - ECCV 2004. ECCV 2004. Lecture Notes in Computer Science, Vol. 3024, pp. 25-36, 2004.
[37] R. Mehran, A. Oyama, and M. Shah, "Abnormal crowd behavior detection using social force model," 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 935-942, 2009.
[38] V. Mahadevan, W. Li, V. Bhalodia, and N. Vasconcelos, "Anomaly detection in crowded scenes," 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1975-1981, 2010.
[39] C. Lu, J. Shi, and J. Jia, "Abnormal Event Detection at 150 FPS in MATLAB," 2013 IEEE International Conference on Computer Vision, pp. 2720-2727, 2013.
[40] W. Li, V. Mahadevan, and N. Vasconcelos, "Anomaly detection and localization in crowded scenes," IEEE Trans Pattern Anal Mach Intell, Vol. 36, No. 1, pp. 18-32, 2014.
[41] D. Xu, Y. Yan, E. Ricci, and N. Sebe, "Detecting anomalous events in videos by learning deep representations of appearance and motion," Computer Vision and Image Understanding, Vol. 156, pp. 117-127, 2017.
[42] J. Kim and K. Grauman, "Observe locally, infer globally: A space-time MRF for detecting abnormal activities with incremental updates," 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2921-2928, 2009.
[43] Y. Cong, J. Yuan, and J. Liu, "Sparse reconstruction cost for abnormal event detection," CVPR 2011, pp. 3449-3456, 2011.
[44] M. Ravanbakhsh, M. Nabi, H. Mousavi, E. Sangineto, and N. Sebe, "Plug-and-Play CNN for Crowd Motion Analysis: An Application in Abnormal Event Detection," 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1689-1698, 2018.
[45] H. Mousavi, M. Nabi, H. Kiani, A. Perina, and V. Murino, "Crowd Motion Monitoring using Tracklet-based Commotion Measure," IEEE International Conference on Image Processing(ICIP), pp. 2354-2358, 2015.
[46] Y. Hao, Y. Liu, J. Fan, and Z. Xu, "Group Abnormal Behaviour Detection Algorithm based on Global Optical Flow," Complexity, Vol. 2021, pp. 12, 2021.
[47] A. Feizi, "Hierarchical detection of abnormal behaviors in video surveillance through modeling normal behaviors based on AUC maximization," Soft Computing, Vol. 24, No. 14, pp. 10401-10413, 2020.
)