LIBRAS Recognition

LIBRAS is the acronym for Brazilian Sign Language, a set of gestures used by people with hearing loss to communicate between them and other people, whether deaf or not.

This was project I did with my peers in high school when I was 15. It was a social call to bring more awareness about the deaf community as well as to study possible technological solutions to overcome communication problems. We held several meetings with our local association for deaf people (ASLA).

For the technical part, I developed a solution to recognize some gestures by counting the number of convexity defects found in the hand. It was a known algorithm used for similar recognitions. The project made use of OpenCV tools. Today, I can see that it was nothing complicated, but back then, it seemed very hard. One of the most challenging aspects was my blockage to learn C++ due to the tight deadline, so I sticked with Unity but had to use a half-baked wrapper. Somehow, barely knowing what I was doing, I managed to pull it off. It was a prototype and recognized very few gestures, but served greatly as a starting point for future discussion and development. We presented it at the local science fair. Unfortunately, though, the judges decided that a potato cannon was a more promising project.

Two interesting takeaways from this project were finding how many people were interested in it because their thesis were similar; and the amount of research that exists today solving the same problem, whether for LIBRAS or ASL. Today techniques are far more advanced and sometimes use hardware to help detect the gestures but the issue raised years ago is still relevant.