Project Overview

Project Summary:

COM-US is an AI-powered communication platform that bridges communication gaps

between Deaf parents and their hearing children (CODAs) through sign language

transformation and interactive dialogue support. In Korea, over 90% of Deaf adults have

hearing children. However, many CODA families struggle with emotional bonding and daily

conversations due to limited shared language.

To address this, COM-US enables Deaf parents to send questions to their children, who

respond using speech or text. These responses are automatically converted into Korean

Sign Language (KSL) grammar using a context-aware language processing system

optimized for sign structure. Key sign words are then matched with official KSL dictionary

videos, helping parents intuitively understand their children’s thoughts and emotions.

Beyond translation, COM-US supports CODA children in naturally acquiring sign language

by mimicking videos and engaging in repeated practice. All conversations are visualized

through “Dialogue Blocks,” encouraging ongoing family interaction and emotional sharing.

By combining accessible technology with cultural sensitivity, COM-US offers more than

just communication—it builds a shared space for learning, empathy, and deeper

connection within CODA families.

Identifying the Challenge

The Social Problem:

CODAs (Children of Deaf Adults) grow up navigating dual linguistic and cultural

environments—sign language at home and spoken language in society. From a young age,

many CODAs act as “young carers,” handling interpretation, paperwork, and

communication with institutions for their Deaf parents. However, Korea lacks institutional

recognition or support for these responsibilities. As a result, CODAs often face emotional

strain and struggle to express their feelings. Most critically, the language gap with their

parents can block open communication, weakening family bonds and affecting identity

development.

Existing sign language tools are limited to word-level translation and don’t support

emotional, sentence-based dialogue. COM-US was created to bridge this gap—

transforming spoken or written input into Korean Sign Language (KSL) grammar and linking

words to official sign videos. This provides Deaf parents and their children with a safe,

structured space to share emotions and thoughts, promoting deeper emotional

connection and more inclusive family communication.

Innovation and Uniqueness

Why Our Project Stands Out:

COM-US is the first emotional communication platform specifically designed for CODA

families. Unlike conventional sign language tools that offer one-way, word-based

translation, COM-US enables two-way, real-time conversations by converting written text

into sentence-level Korean Sign Language (KSL) grammar.

The system provides accurate and context-aware translations, which are then paired with

sign videos from the official KSL dictionary. This ensures both linguistic accuracy and

intuitive communication. A unique “Sign Practice” feature allows children to mimic sign

videos, making sign language learning engaging and natural. The platform also visualizes

family conversations through a “Dialogue Block” interface, encouraging continuous

emotional sharing.

What sets COM-US apart is its emotion-focused design: it’s not just a tool for translation,

but a bridge that fosters empathy, understanding, and identity development within CODA

families—transforming everyday dialogue into lasting emotional connection.

Insights and Development

Learning Journey:

We initially focused on building a sign language education tool, but as development

progressed, we realized that instruction alone couldn’t address the deeper

communication issues faced by CODA families. To better understand these challenges,

our team studied Korean Sign Language (KSL) syntax, and some members completed a 3-

month education program. We also shared weekly insights to grasp CODAs’ emotional

experiences better.

Interviews revealed that many CODAs face emotional distance, difficulty expressing

feelings, and psychological burdens from acting as mediators. These insights led us to

shift our project toward emotional exchange rather than simple instruction. This journey

taught us to ask what users truly need. Understanding the root of the problem became the

foundation for every decision, reminding us that meaningful technology must begin with

empathy and deep user understanding.

Development Process:

COM-US is an AI communication platform built on Android, Spring Boot, and Flask. We

developed a custom AI pipeline that transforms user input into Korean Sign Language (KSL)

grammar and links it to appropriate sign videos. Our biggest challenge was generating

natural KSL sentences. To solve this, our team studied Korean Sign Language syntax and

designed GPT prompts that applied grammatical rules such as word order rearrangement,

negation handling, and sentence segmentation. We also addressed the limited vocabulary

of the o`icial KSL dictionary by building a synonym expansion system. This allowed us to

match out-of-vocabulary words with semantically similar ones and show corresponding

sign videos. Through iterative testing and real user feedback, we refined our system into a

grammar-aware, meaning-centered sign language pipeline that supports e`ective and

emotionally expressive communication.