My grandmother’s house stands on a narrow one-way street in Japan. For as long as I
can remember, every summer and winter break, I opened the wooden windows of Aki’s house to the sound of cicadas or snowfall. I also set a warm bowl of rice and water for my grandpa’s altar in the morning. Of course, there were many days I did this routine unwillingly, but eventually as the years passed, the habit began to take over. Before I realized it, I had started to open the windows and set the bowl down unconsciously, without even thinking about it. This routine was what represented the time I spent with my grandma.
The last two summers, however, I spent all of summer and winter in just one place: a long-term
rehabilitation home. My Aki had suffered a stroke, and I spent entire days taking care of her. It was not a typical idea of a holiday, yet I never gave it a single thought. I’m sure that Aki never
thought of anything when she had her grandchildren come and intrude in her household all summer when I was young. The days were spent as a mirror of roles reversed. When I was young, Aki delighted in teaching me all the things I know now. Aki’s stroke left her half-paralyzed on one side, which made speech difficult. Now I was teaching her how to speak again. And that’s exactly how I wanted to spend my summer.
“During summer research at George Mason University, I created an American Sign Language (ASL) recognition device with graduate student mentors.”
- Arisa C.
Her difficulty with speech and recovery inspires my current academic interest. Human communication comprises gestures, postures, and facial expressions that all complement one another to deliver a message. After taking an active part in Aki’s speech therapy sessions, I began to pay attention to the importance of language. That’s how I got attracted to natural language processing (NLP): connecting humans, machines, and languages through computational methods. This perfectly combined my academic pursuit of computer science (CS) and my newfound interest in linguistics.
During summer research at George Mason University, I created an American Sign Language (ASL) recognition device with graduate student mentors. I spent that summer captivated by NLP, transforming my visual dances of ASL to acceleration and gyroscopic measurements. I then used this data to mold training sets for my Decision Tree machine learning algorithms. Each flick of our signers’ wrists traveled down the branches of our decision trees, separated by
if-else conditions based on numerical measurements from our sensors. I even noticed that everyone signs ASL a little differently, discovering a universe of non-verbal accents that I hadn’t
known was possible.
"As I learn more about CS and linguistics, I hope to one day join the cycle of humans improving machines and machines, in turn, enhancing our quality of life."
- Arisa C.
As our project progressed, our training sets needed deeper (the same person signing the same
sign multiple times) and more varied (multiple people signing the same sign) sets of data. We
expanded our pool of ASL signers, improving our training sets so our algorithms could translate
ASL signs into written text with 97.9% accuracy.
This made me wonder: if our physical ASL movements could be reshaped into the machine language of ones and zeros and then transposed into English text, what further possibilities did
interconnecting the humanities and STEM hold? Could it also be applied to speech therapy for
stroke patients, like my Aki, to provide real-time automated feedback to motivate and monitor the patients’ efforts?
How can a machine better comprehend context, slang, emotion, and sarcasm—traits that aren’t
so clear-cut like the if-else statements in my algorithms? More specifically, how do we as
programmers improve these machines to increase their capabilities?
Studying new algorithms has opened my eyes to NLP’s boundary-breaking power. I’m excited
that my code can bring us closer to machines that think, reason, and communicate. If I learn enough about algorithms that can analyze human language, I can work on the Alexas and the Google Searches of tomorrow — applying the entirety of my knowledge, experience, and passion to building computers that can better interpret and understand us. As I learn more about CS and linguistics, I hope to one day join the cycle of humans improving machines and machines, in turn, enhancing our quality of life.
I want to contribute to the next evolutionary step of language, where we maximize the potential of communication using machines. Through NLP, we can accomplish more than what we can achieve as just humans or just devices. The future is technology, and I’m excited to be a bridge between humans and machines through my twin passions of linguistics and CS.
- Arisa C.