Project Overview
Project Summary:
COM-US is an AI-powered communication platform that bridges communication gaps
between Deaf parents and their hearing children (CODAs) through sign language
transformation and interactive dialogue support. In Korea, over 90% of Deaf adults have
hearing children. However, many CODA families struggle with emotional bonding and daily
conversations due to limited shared language.
To address this, COM-US enables Deaf parents to send questions to their children, who
respond using speech or text. These responses are automatically converted into Korean
Sign Language (KSL) grammar using a context-aware language processing system
optimized for sign structure. Key sign words are then matched with official KSL dictionary
videos, helping parents intuitively understand their children’s thoughts and emotions.
Beyond translation, COM-US supports CODA children in naturally acquiring sign language
by mimicking videos and engaging in repeated practice. All conversations are visualized
through “Dialogue Blocks,” encouraging ongoing family interaction and emotional sharing.
By combining accessible technology with cultural sensitivity, COM-US offers more than
just communication—it builds a shared space for learning, empathy, and deeper
connection within CODA families.
Identifying the Challenge
The Social Problem:
CODAs (Children of Deaf Adults) grow up navigating dual linguistic and cultural
environments—sign language at home and spoken language in society. From a young age,
many CODAs act as “young carers,” handling interpretation, paperwork, and
communication with institutions for their Deaf parents. However, Korea lacks institutional
recognition or support for these responsibilities. As a result, CODAs often face emotional
strain and struggle to express their feelings. Most critically, the language gap with their
parents can block open communication, weakening family bonds and affecting identity
development.
Existing sign language tools are limited to word-level translation and don’t support
emotional, sentence-based dialogue. COM-US was created to bridge this gap—
transforming spoken or written input into Korean Sign Language (KSL) grammar and linking
words to official sign videos. This provides Deaf parents and their children with a safe,
structured space to share emotions and thoughts, promoting deeper emotional
connection and more inclusive family communication.
Innovation and Uniqueness
Why Our Project Stands Out:
COM-US is the first emotional communication platform specifically designed for CODA
families. Unlike conventional sign language tools that offer one-way, word-based
translation, COM-US enables two-way, real-time conversations by converting written text
into sentence-level Korean Sign Language (KSL) grammar.
The system provides accurate and context-aware translations, which are then paired with
sign videos from the official KSL dictionary. This ensures both linguistic accuracy and
intuitive communication. A unique “Sign Practice” feature allows children to mimic sign
videos, making sign language learning engaging and natural. The platform also visualizes
family conversations through a “Dialogue Block” interface, encouraging continuous
emotional sharing.
What sets COM-US apart is its emotion-focused design: it’s not just a tool for translation,
but a bridge that fosters empathy, understanding, and identity development within CODA
families—transforming everyday dialogue into lasting emotional connection.
Insights and Development
Learning Journey:
We initially focused on building a sign language education tool, but as development
progressed, we realized that instruction alone couldn’t address the deeper
communication issues faced by CODA families. To better understand these challenges,
our team studied Korean Sign Language (KSL) syntax, and some members completed a 3-
month education program. We also shared weekly insights to grasp CODAs’ emotional
experiences better.
Interviews revealed that many CODAs face emotional distance, difficulty expressing
feelings, and psychological burdens from acting as mediators. These insights led us to
shift our project toward emotional exchange rather than simple instruction. This journey
taught us to ask what users truly need. Understanding the root of the problem became the
foundation for every decision, reminding us that meaningful technology must begin with
empathy and deep user understanding.
Development Process:
COM-US is an AI communication platform built on Android, Spring Boot, and Flask. We
developed a custom AI pipeline that transforms user input into Korean Sign Language (KSL)
grammar and links it to appropriate sign videos. Our biggest challenge was generating
natural KSL sentences. To solve this, our team studied Korean Sign Language syntax and
designed GPT prompts that applied grammatical rules such as word order rearrangement,
negation handling, and sentence segmentation. We also addressed the limited vocabulary
of the o`icial KSL dictionary by building a synonym expansion system. This allowed us to
match out-of-vocabulary words with semantically similar ones and show corresponding
sign videos. Through iterative testing and real user feedback, we refined our system into a
grammar-aware, meaning-centered sign language pipeline that supports e`ective and
emotionally expressive communication.
