CMU App

A team of students at Carnegie Mellon University including Indian American students Sashank Gogula (second from right) and Mehar Sawhney (right), has developed a free mobile app that uses visual cues to help children and adults form sentences. Also seen here are (from left) DFA@CMU members Elizabeth Board, Elizabeth La, and Alyse Baker, a Pittsburgh speech-language pathologist who developed the language creation aid that the students have adapted as an iPad app. (cmu.edu photo)

Students in the Carnegie Mellon University chapter of Design for America, which include Indian Americans Sashank Gogula and Mehar Sawhney, have developed a free mobile app that uses visual cues to help children and adults — particularly those with language difficulties — form meaningful sentences that are grammatically correct, the university said in a Sept. 18 report.

The education app, called Sentence Mosaics, uses photos and color-coded parts of speech to prompt users as they construct sentences, it said.

The app is useful for elementary students and for a wide range of children and adults who are on the autism spectrum, or who have developmental disabilities, hearing disorders, stroke-related language deficits, dementia and other problems with language, said Alyse Baker, a certified speech-language pathologist who has honed the system for decades, the report added.

"Alyse has used colored blocks, colored cards and foam tiles attached with Velcro to a foam board as aids, but that's not practical for use outside the classroom," said Sawhney, a DFA@CMU member and a senior majoring in mathematics and physics, in the report. "We worked with her to translate her system into an iPad app that can be taken anywhere."

Baker developed Sentence Mosaics based on longstanding research and practice within the field of speech-language pathology. She has long recognized that transforming the system into an app would make sense, but found the prospect daunting, the university report said.

David Kosbie, an associate teaching professor of computer science at CMU, connected Baker two years ago with DFA@CMU, an interdisciplinary group of students who design and engineer solutions for social good.

A group of DFA members, including Sawhney, computer science majors Elizabeth Board and Gogula, and cognitive science major Elizabeth La, worked with Baker to translate physical aids into digital ones. Jennifer Chou, who graduated last spring with a computer science degree, also was a major contributor to the project, the report said.

Photos are key to the system. Users write sentences that describe actions depicted in photos, preferably photos that include the student or show a scene from their everyday life. Board said the app can import photos from an iPad or iPhone, it added.

The app is designed to be used with a teacher, parent or other "coach," who helps by loading photos into the app and by designating a string of colored boxes that signal to the student which parts of speech — nouns, pronouns, verbs, adverbs, adjectives, etc. — are necessary to describe the photo, the report added.

(0) comments

Welcome to the discussion.

Keep it Clean. Please avoid obscene, vulgar, lewd, racist or sexually-oriented language.
PLEASE TURN OFF YOUR CAPS LOCK.
Don't Threaten. Threats of harming another person will not be tolerated.
Be Truthful. Don't knowingly lie about anyone or anything.
Be Nice. No racism, sexism or any sort of -ism that is degrading to another person.
Be Proactive. Use the 'Report' link on each comment to let us know of abusive posts.
Share with Us. We'd love to hear eyewitness accounts, the history behind an article.