Blog of the Data Science for the Digital Society research group. Digital Society Innovation, Applied Artificial Intelligence, data analysis and smart living and business.

05 October 2021 | Posted by Editorial Team DS4DS

Sign language processing and its conversion to text

This Monday, 4th October, 2021, the student Mar Galiana, tutored by Prof. Dr. Elisabet Golobardes and MSc. Nuria Valls. Work in which it has been wanted to implement a predictive model that uses images as input data. These images refer to thirty-eight static American Sign Language (ASL) gestures. The gestures belong to the English alphabet and to the numbers from zero to ten.

There have been two predictive models used: decision trees and neural networks. The Extreme Gradient Boosting algorithm was used with the decision tree, obtaining an accuracy of 100 %. It was eventually discarded because, as the number of samples in the dataset increased, its time cost increased too much to be used under this project execution conditions.

As far as neural networks are concerned, two types were implemented: one artificial and one convolutional. Two types of classifier were used: multi-class and binary, using the One vs All strategy.

The best results were obtained with the multi-class convolutional neural network, achieving an accuracy of 99.52 % with a total execution time of 1 hour, 14 minutes and 49 seconds, and a memory cost of 15,594.4 GB.


Add new comment

This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
17 + 1 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.