A Review on Deep Learning Based Sign Language Recognition Translation

Junkai Huang

doi:10.62051/ijcsit.v4n1.04

Authors

Junkai Huang

DOI:

https://doi.org/10.62051/ijcsit.v4n1.04

Keywords:

Hand Gesture Recognition, Sign Languages, Deep Learning Techniques, Performance Metrics

Abstract

Hand posture is often referred to as gesture. It refers to the specific movements and postures that occur when a person utilizes his or her arms. It is one of the earliest and still widely used communication tools. In general, gestures are both dynamic and static. In the process of long-term social practice, gestures are given a variety of specific meanings, has a rich expressive power, the range of motion is extensive when the fingers, wrists, elbows, shoulders, and other joints of the hand are considered. The capacity for flexibility is considerable. Gestures have become a primary means of expression for humans, enabling the conveyance of emotions. In this context, body language occupies a pivotal role. Sign language is defined as the use of hand gestures with or without the use of facial expressions to convey meaning. The combination of hand gestures and facial expressions can be used to imitate images or to represent syllables, which together constitute a certain meaning or word. Sign language is a means of communication for individuals who are deaf or hard of hearing. It is an essential tool for communication, enabling them to express themselves and interact with others in a way that is accessible and understandable. This paper summarizes the deep learning based gesture recognition techniques in recent six years. In recent years, gesture recognition has been in the hot spot, and every year the articles on gesture recognition are constantly updated, so it is evident that a review of the research on gesture recognition is very necessary. This paper also compares and analyzes the existing gesture recognition methods and analyzes their advantages and disadvantages, and puts forward an outlook on the future development.

Downloads

Download data is not yet available.

References

[1] HINTON G E, OSINDERO S, and TEH Y W. A fast learning algorithm for deep belief nets [J]. Neural Computation, 2006, 18(7): 1527–1554. doi: 10.1162/neco.2006.18.7.1527.

[2] Snehal Abhijeet Gaikwad, Dhananjay Upasani, Virendra Shete, “Review and Trends on Hand Gesture Recognition of Sign Language based on Deep Learning Approaches” IEEE Xplore Part Number: CFP23BC3-ART; ISBN: 978-1-6654-5630-2.

[3] H. Cooper, B. Holt, and R. Bowden, “Sign language recognition,” in Visual Analysis of Humans. London, U.K.: Springer, 2011, pp. 539–562.

[4] S. Skaria, A. Al-Hourani, M. Lech and R. J. Evans, "Hand-Gesture Recognition Using Two-Antenna Doppler Radar With Deep Convolutional Neural Networks," in IEEE Sensors Journal, vol. 19, no. 8, pp. 3041-3048, 15 April15, 2019, doi: 10.1109/JSEN.2019.2892073.

[5] LIAO Yanqiu, XIONG Pengwen, MIN Weidong, et al.Dynamic sign language recognition based on video sequence with BLSTM-3D residual networks [J]. IEEE Access, 2019, 7: 38044–38054. doi: 10.1109/ACCESS.2019.2904749.

[6] Xiao Q, Qin M, Guo P, et al. Multimodal fusion based on LSTM and a couple conditional hidden Markov model for Chinese sign language recognition [J]. IEEE Access, 2019, 7: 112258-112268.

[7] Luo Yuan, Li Dan, Zhang Yi. Chinese sign language recognition based on spatio-temporal attention network [J]. Semiconductor Photonics, 2020, 41 (03) 414-419. DOI: 10. 16818 /j.issn1001-5868.2020.03.021.

[8] S. Aly and W. Aly, "DeepArSLR: A Novel Signer-Independent Deep Learning Framework for Isolated Arabic Sign Language Gestures Recognition," in IEEE Access, vol. 8, pp. 83199-83212, 2020, doi: 10.1109/ACCESS.2020.2990699.

[9] Safa Ameur, Anouar Ben Khalifa, Med Salim Bouhlel, A novel hybrid bidirectional unidirectional LSTM network for dynamic hand gesture recognition with Leap Motion, Entertainment Computing, Volume 35, 2020, 100373, ISSN 1875- 95 21, https://doi.org/10.1016/j.entcom.2020.100373.

[10] Dushyant Kumar Singh, 3D-CNN based Dynamic Gesture Recognition for Indian Sign Language Modeling, Procedia Computer Science, Volume 189, 2021, Pages 76-83, ISSN 1877- 050 9, https://doi.org/10.1016/j.procs.2021.05.071.

[11] Sahoo, Jaya Prakash, Allam Jaya Prakash, Paweł Pławiak, and Saunak Samantray. 2022. "Real-Time Hand Gesture Recognition Using Fine-Tuned Convolutional Neural Network" Sensors 22, no. 3: 706. https://doi.org/10.3390/s22030706

[12] Bhaumik, G., Verma, M., Govil, M.C. et al. ExtriDeNet: an intensive feature extrication deep network for hand gesture recognition. Vis Comput 38, 3853–3866 (2022). https://doi.org/10.1007/s00371-021-02225-z

[13] H. Wu et al., "CvT: Introducing Convolutions to Vision Transformers," 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada, 2021, pp. 22-31, doi: 10.1109/ICCV48922.2021.00009.

[14] Y. Altaf, A. Wahid and M. M. Kirmani, "Deep Learning Approach for Sign Language Recognition Using DenseNet201 with Transfer Learning," 2023 IEEE International Students' Conference on Electrical, Electronics and Computer Science (SCEECS), Bhopal, India, 2023, pp. 1-6, doi: 10.1109/SCEECS57921.2023.10063044.

[15] J. Huang, J. Chaijaruwanich and V. Chouvatut, "Video-based Sign Language Recognition with R (2+1)D and LSTM Networks," 2024 16th International Conference on Knowledge and Smart Technology (KST), Krabi, Thailand, 2024, pp. 214-219, doi: 10.1109/KST61284.2024.10499646.

[16] S. Kankariya, K. Thakre, U. Solanki, S. Mali and A. Chunawale, "Sign Language Gestures Recognition using CNN and Inception v3," 2024 International Conference on Emerging Smart Computing and Informatics (ESCI), Pune, India, 2024, pp. 1-6, doi: 10.1109/ESCI59607.2024.10497401.