Volume 11, Number 14, September 2021

Mask Region-Based Convolutional Neural Networks (R-CNN) for Sinhala Sign Language to Text Conversion

  Authors

R. D. Rusiru Sewwantha¹ and T. N. D. S. Ginige², ¹University of Central Lancashire, Sri Lanka, ²Universal College Lanka, Sri Lanka

  Abstract

Sign Language is the use of various gestures and symbols for communication. It is mainly used by disabled people with communication difficulties due to their speech or hearing impediments. Due to the lack of knowledge on sign language, natural language speakers like us, are not able to communicate with such people. As a result, a communication gap is created between sign language users and natural language speakers. It should also be noted that sign language differs from country to country. With American sign language being the most commonly used, in Sri Lanka, we use Sri Lankan/Sinhala sign language. In this research, the authors propose a mobile solution using a Region Based Convolutional Neural Network for object detection to reduce the communication gap between the sign users and language speakers by identifying and interpreting Sinhala sign language to Sinhala text using Natural Language Processing (NLP). The system is able to identify and interpret still gesture signs in real-time using the trained model. The proposed solution uses object detection for the identification of the signs.

  Keywords

Object detection, TensorFlow, Mask R-CNN, Sinhala, Sign Language, VideoCapture.