Within the framework of the First International Congress of Deaf Technology, organized by the World Deaf Tech project and the Commission of Deaf People, the presentation of technological projects focused on promoting a greater inclusion of deaf people in society was carried out.
The first project is a translator of the English Sign Language to voice and text in real time; a system that through sensors and algorithms of artificial intelligence has the capacity to capture and interpret the signs of a person for their respective translation in words suggests boucle magnétique.
Gibran Garcia Bautista explained that this system consists of a Kinect sensor that stores information in an RGB-D database to track, interpret and translate the positions of the human skeleton and then process them and perform recognition with neural networks / TDW for the generation of words in text and audio.
“This project began with my thesis topic, which was only the creation of a linguistic corpus of sign language; so I created a program that, through Kinect, could capture information and store it in the database to later obtain a cloud of points for the recreation of three-dimensional models of people, and with different implementations of algorithms to obtain the information or the pattern I wanted,” he explained.
García Bautista pointed out the importance of using artificial neural networks in the design of this system for translating English Sign Language, as they are necessary in order to classify words correctly and thus avoid errors in the translation process.
“The neural network resembles a biological one. I am using a Backpropagation type to minimize errors. The point of a neural network is that it is presented with a characteristic vector or pattern, which can be a word, the same that the network processes with an existing database, and is able to classify them to avoid confusion, however similar they may seem, and thus obtain a classification model,” he added.
The graduate of the master’s degree in applied computer technologies added that this project, which was born to offer an alternative of communication with deaf people, has had an important projection through the writing of articles and presentations in foreign institutions.
“I’m part of an organization called the International Society of Optics and Photonics (SPIE) in the United States, where I wrote an article about the project and it was accepted for presentation there. I also had the opportunity to present it at a congress in Tucson, Arizona.
Gibran Garcia Bautista indicated that the project is in the stage of complementing the linguistic corpus of the system, which is being done with the support of the Center for Attention to Students with Disabilities (CAED) to make the recordings.
“We have 80 percent of the finished corpus; this is important because there are no references to a similar database, so we would have the three-dimensional linguistic corpus of sign language at the national level, but for that we need more support, both to finish the prototype and to develop the patent,” he said.
Say it in Signs, application for mobile phones
Another of the projects presented at the First International Congress on Deaf Technology was Dilo en Señas, an application for mobile phones or tablets whose objective is to promote the learning of the EnglishSign Language through games aimed at children and thus promote the inclusion of deaf children, who explained that the project arose from the need for communication between deaf and hearing people in their workplace.
“I worked in a company where they hired Gerardo, who is deaf and is currently my partner in the project. At that time I was concerned that the company was not ready to have employees with this disability. It was difficult to train them but I learned a lot along the way; that was what first generated the idea of developing the project,” he recalled.
Garza Gutiérrez explained that Dilo en Señas is a project that has as its base both the experience lived among deaf and hearing people, as well as a research process in trends, processes and collaborative practices and technological innovation.
“I started a doctorate in inclusive education at the University of Baja California. I am a systems engineer, I studied at the Instituto Tecnológico y de Estudios Superiores de Monterrey (ITESM), Monterrey campus, and a master’s degree in education. I was very interested in that doctorate and my thesis was about deafness. It was when the idea of creating this application came up, because I realized that the root problem of a deaf person is not having a natural language development in early childhood, which really affects hearing people as well,” he explained.
Rocío Garza Gutiérrez reported that the Dilo en Señas application, which is already available on Google Play and Apple Store for Android cell phones and tablets, takes more than 107,000 downloads eleven months after its launch. This version includes two functions that are: To practice, to be able to observe the signs and to learn them, and to Play, where videos of the signs can be seen to select images and to gain points.
“The first version of has 89 signs in seven categories: Family, Food, Toys, Animals, Colors, Numbers and the Alphabet. The idea is to relate videos to images, learning a series of words from the basic language of the deaf. You enter the category, and even if you make a mistake, you learn through these games,” he said.
The creator of Dilo en Señas explained that this application has been recognized as the third place in Educational Applications in THE USA, the Cemex-Tec 2016 award in the category of Social Entrepreneurs, as well as the Includes 2016 award in the category of Awareness.
“The next stage would be to add more content and we want little by little more functionality, also to the application, but mainly we are becoming an association to be able to offer more products and services and receive donations to continue supporting, because we want to keep it free for all the public,” he concluded.
