![]()
FNDC Summer • 2019
16
Many deaf children lack early access to American Sign Language.
This woman is harnessing tech to change that.
From apps to digital books, Melissa Malzkuhn is dedicated to making ASL accessible — and fun.
https://www.thelily.com/many-deaf-children-lack-early-access-to-american-sign-language-this-woman-is-harnessing-tech-to-change-that/
Posted: April 15 2019
Growing up, Melissa Malzkuhn cherished story time,
squeezing on a couch or bed with her two siblings
while their mom read them their favorite books.
Following along in the Malzkuhn household went
beyond looking at words and pictures on the page.
Like their mother, the Malzkuhn children were born
deaf. As she ipped through the pages, Malzkuhn’s
mother read the stories “aloud” using American Sign
Language.
“My mom would just open up a book and just start
signing the book,” Malzkuhn said.
Malzkuhn's mom did the same with TV, translating
captions in real time as the kids watched cartoons or
movies. The exposure paid o. Before long, Malzkuhn
was able to read on her own.
Malzkuhn’s experience and early exposure to
signing, and therefore reading, is unfortunately far
from the norm. As the third generation in her family
born without the ability to hear, she grew up learning
American Sign Language. But more than90 percent
of deaf children are born to hearing parents. Many of
those children lack early access to ASL, a language
used by hundreds of thousands of members of the
deaf community that is celebrated every April 15
onNational ASL Day. As a result, many deaf children
experiencea language decit early on that can set
them behind on learning and literacy.
“Language deprivation really means that someone is
set on a path of continued struggle, they’re always
catching up,” Malzkuhn, who signs, said through a
translator. “They’re missing a lot.”
Malzkuhn doesn’t think that should be the case. The
37-year-old advocate, artist anddigital strategisthas
spent the last decade-plus leveraging her passion for
art, technology and creative storytelling to increase
exposure to ASL and, hopefully, close that gap.
These days, that work is happening inMotion Light
Lab, a research center Malzkuhn runs at Gallaudet
University in Washington, D.C. Core to its mission is
using the power of visuals to improve learning and
language for the deaf community — “The lion’s share
of information comes in through the eyes.
How does that impact cognitive processes?” she said
of the question driving the center’s work — and even
the building that houses the lab reects that. It’s airy,
with wide hallways and open spaces, and the hallway
leading to the lab is lined with colorful posters. Those
posters depict the cover art for a series of digital
children’s books Malzkuhn built for iPads so families
can follow along as actors sign the words. Every
aspect of the digital books, from the technology
to the illustrations and storylines, was specically
designed with a deaf audience in mind.
“Our whole approach was very organic,” she said.
“How do you create the reading experience for deaf
children? How do you create an experience that’s
bilingual and seamless visually? And then we also
needed to develop the actual story.”
The books, which have been translated into multiple
languages, have been a hit among both kids and
their parents.
“A lot of parents will struggle with feeling like they
don’t sign well enough. And we have to tell them,
like, actually it’s okay. It’s okay even if you don’t
sign perfectly,” she said. “This has turned out to be
an amazing way for parents to feel a little bit more
condent about signing with their kid. They’re able
to point things out and still engage, and there’s that
reading experience.”
Malzkuhn is now building on the success of that
concept with projects like anapp featuring nursery
rhymes signed by cartoon avatars and new exper-
iments involving articial intelligence and early
learning. The work to make those ideas a reality
happens inside the one-room lab, which looks like
a cross between a classroom and a commercial
studio. Chairs and tables are mixed in with photo
lights and giant monitors. A black track suit covered
in motion sensors, which Malzkuhn and colleagues
use to record the movements they’ll turn into signing
avatars, hangs on a mannequin. No fewer than 16
cameras are mounted around the room to catch the
gesturing from every angle.
“Our goal is to create signing characters, and we
need to gure out a technology that can do that, and
do that in 3-D,” she said. “We need to see what that
looks like and the complexity of signing, like how
much detail do you need in the ngers? How much
do you need to use the right facial expressions?”
While promoting early learning and literacy is a
central focus of Malzkuhn’s work, her products
aren’t just for kids. Over the years, she’s launched an
award-winning iPhone ASL app that teaches users
how to use ASL to communicate common phrases
(“You’re cute,” “That’s a cool shirt”); she created GIFs
that translate key Washington, D.C., words (“The
Capitol,” “The monument”); and she brought ASL to
group chats everywhere with a series of Apple iOS
stickers signed by America’s Top Model alum Nyle
DiMarco. The stickers include slang like “Omg” and
“Lol.” High demand has led Malzkuhn and her team
to add even more vocabulary categories to the ASL
app since its initial launch.
“We found that our audience ended up being so
much wider than we had ever expected,” she said.
Malzkuhn’s groundbreaking initiatives have already
snagged her time on a Ted Talk stage and a spot
in the Obama Foundation’s
inaugural two-year
fellowship class. But her work is far from over. The
lab is now experimenting with 3-D avatars, articial
intelligence and more cutting-edge technology to
further its mission. She just launched a new edition
in her children’s book series. And a recent grant from
Mitsubishi Electric America Foundation will allow her
to spend the next three years training members of
the deaf community to make storybook apps of their
own.
In recent years, Malzkuhn’s commitment to early
exposure to language and literacy for deaf children
extends well beyond work hours. After her own son
was born deaf, she found herself bringing her exper-
iments into his life. He loves the apps and books his
mom created. And, of course, she’s continuing the
tradition of interpreting shows and story time at
home.
(Courtesy of Melissa Malzkuhn)
