A Novel Approach to Communicate with Video Game Character using Cascade Classifiers

Mohammadzadeh, M.; Khosravi, H.

doi:10.22044/jadm.2020.9788.2110

Document Type : Original/Review Paper

Authors

Faculty of Electrical Engineering, Shahrood University of Technology, Shahrood, Iran.

https://doi.org/10.22044/jadm.2020.9788.2110

Abstract

Today, video games have a special place among entertainment. In this article, we have developed an interactive video game for mobile devices. In this game, the user can control the game’s character by his face and hand gestures. Cascading classifiers along with Haar-like features and local binary patterns are used for hand gesture recognition and face detection. The game’s character moves according to the current hand and face state received from the frontal camera. Various ideas are used to achieve the appropriate accuracy and speed. Unity 3D and OpenCV for Unity are employed to design and implement the video game. The programming language is C#. This game is written in C# and developed for both Windows and Android operating systems. Experiments show an accuracy of 86.4% in the detection of five gestures. It also has an acceptable frame rate and can run at 11 fps and 8 fps in Windows and Android respectively.

Keywords

20.1001.1.23225211.2021.9.2.9.2

References

[1] Sarkar S, P.V., and Chellappa R. "Deep Feature-based Face Detection on Mobile Devices". in 2016 IEEE International Conference on Identity, Security and Behavior Analysis (ISBA). Sendai, Japan: IEEE. 2016.

[2] Chen Q, G.N., and Petriu E. "Real-time Vision-based Hand Gesture Recognition Using Haar-like Features". in 2007 IEEE Instrumentation & Measurement Technology Conference IMTC 2007. Warsaw, Poland: IEEE. pp. 1-6, 2007.

[3] Khodabakhsh, V. and M. Islami. "Detection of hand movements for human-computer interaction using image processing techniques (in Farsi)". in Second International Conference of Researchers in Computer Engineering and Information Technology. Tehran, Iran. 2017.

[4] Qingtang L, Y.W., Linjing W, Jingxiu H, and Peng W. "Design and Implementation of a Serious Game Based on Kinect". in 2015 International Conference of Educational Innovation through Technology (EITT). Wuhan, China: IEEE. pp. 13-18, 2015.

[5] Zhu Y, a.Y.B. "Real-Time Hand Gesture Recognition with Kinect for Playing Racing Video Games". in 2014 International Joint Conference on Neural Networks (IJCNN). Beijing, China: IEEE. pp. 3240-3246, 2014.

[6] Farzin Nezhad, F. and J. Rasti. "A computer-sports game for practicing forehand drive shading using Kinect Xbox technology (in Farsi)". in First International Conference on Opportunities and Challenges Computer Games. Isfahan, Iran. 2017.

[7] Elrefaei L, A.A., Alamoudi H, Almutairi S, and Al-rammah F. "Real-time face detection and tracking on mobile phones for criminal detection". in 2017 2nd International Conference on Anti-Cyber Crimes (ICACC). Abha, Saudi Arabia: IEEE. 2017.

[8] Viola, P. and M. Jones. "Rapid object detection using a boosted cascade of simple features". in Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001. IEEE. pp. 511-518, 2001.

[9] Mao Q, S.H., Liu Y, and Jia R, "Mini-Yolov3: Real-Time Object Detector for Embedded Applications". IEEE Access. 7, pp. 133529 - 133538, 2019.

[10] Nanda, M., You Only Gesture Once (YouGo): American Sign Language Translation using YOLOv3. 2020, Purdue University Graduate School.

[11] Fang W, W.L., and Ren P, "Tinier-YOLO: A Real-Time Object Detection Method for Constrained Environments". IEEE Access. 8, pp. 1935-1944, 2020.

[12] Karabasi, M., Z. Bhatti, and A. Shah. "A model for real-time recognition and textual representation of malaysian sign language through image processing". in Proceedings of the 2013 International Conference on Advanced Computer Science Applications and Technologies. Kuching, Malaysia. pp. 195–200, 2013

[13] Lahiani, H., M. Elleuch, and M. Kherallah. "Real time hand gesture recognition system for android devices". in 2015 15th International Conference on Intelligent Systems Design and Applications (ISDA). IEEE. pp. 591-596, 2015.

[14] Prakash, J.G., U.K., "Hand Gesture Recognition". International Journal of Recent Technology and Engineering. 7, pp. 54-59, 2019.

[15] Mazzia V, K.A., Salvetti F, and Chiaberge M, "Real-Time Apple Detection System using Embedded Systems With Hardware Accelerators: An Edge Ai Application". IEEE Access. 8, pp. 9102 - 9114, 2020.

[16] Rautaray, S.S. and A. Agrawal. "Interaction with virtual game through hand gesture recognition". in 2011 International Conference on Multimedia, Signal Processing and Communication Technologies. Aligarh, India: IEEE. pp. 244-247, 2011.

[17] Hosseini, M.M. and J. Hassanian, "Applying mean shift and motion detection approaches to hand tracking in sign language". Journal of AI and Data Mining. 2(1), pp. 15-24, 2014.

[18] Anitha, J., G. Mani, and K.V. Rao, Driver drowsiness detection using viola jones algorithm, in Smart Intelligent Computing and Applications. 2020, Springer. p. 583-592.

[19] Barnouti, N.H., et al., "Face detection and recognition using Viola-Jones with PCA-LDA and square euclidean distance". 7(5), pp. 371-377, 2016.

[20] Ren, J., N. Kehtarnavaz, and L. Estevez. "Real-time optimization of Viola-Jones face detection for mobile platforms". in 2008 IEEE Dallas Circuits and Systems Workshop: System-on-Chip-Design, Applications, Integration, and Software. IEEE. pp. 1-4, 2008.

[21] Vikram, K. and S. Padmavathi. "Facial parts detection using Viola Jones algorithm". in 2017 4th International Conference on Advanced Computing and Communication Systems (ICACCS). IEEE. pp. 1-4, 2017.

[22] Ojala, T., M. Pietikäinen, and D. Harwood, "A comparative study of texture measures with classification based on featured distributions". Pattern Recognition. 29(1), pp. 51-59, 1996.

A Novel Approach to Communicate with Video Game Character using Cascade Classifiers

References

References

Volume 9, Issue 2April 2021Pages 227-234

Volume 9, Issue 2
April 2021
Pages 227-234