Context
What if we could use technology to enable people to communicate with each other no matter what language they spoke? For this project, we demonstrated a use case of how Machine Learning technology could be used to break down language barriers and build connections with people. We prototyped an accessible service that translates American Sign Language to voice and speech to text.
What is it?
NeuroPreter is a sign-language interpreter powered by the ever-improving Neural Network algorithms. The product is designed to facilitate a conversation between people who are hearing-impaired and those who are enabled.
How does it work?
NeuroPreter translates sign language into conversational speech, and speech-to-text to enable a conversation. It is set up as a communication-station where people who are hearing impaired and enabled could come and talk to each other, much like in a coffee-shop. The product eliminates the need for people to learn American Sign Language (ASL) and still be able to communicate with the hearing-impaired, and develop closer relationships.
How was it made?
NeuroPreter learns the sign language gestures using machine learning. It uses Wekinator and OpenFrameworks to classify various sign-gestures. A video camera is used to capture the live feed of the person talking in ASL. A Processing script is then used to convert the interpreted gestures into an audio output to mimic conversational speech. On the other side, a Javascript enabled browser is employed to convert speech to text for the hearing-impaired user to read.
This project was done as part of a one-week course on Machine Learning, along with my teammates Chaeri Bong and Shalin Shah at CIID. The brief was to create an interactive machine learning product using simple supervised or unsupervised learning techniques.
