2002 Conference Proceedings

Go to previous article 
Go to next article 
Return to 2002 Table of Contents


ENGLISH TO SIGN LANGUAGE TRANSLATION SOFTWARE

Carol J. Wideman
President & CEO
Vcom3D, Inc
3452 Lake Lynda Drive, Suite 260
Orlando, FL 32817
Tel: 407.737.4695
Email: CarolW@vcom3d.com

The Need

Research indicates that the median reading comprehension of seventeen and eighteen year old Deaf and Hard-of-Hearing students is at the fourth grade level (Holt, et al., 1997) (Gallaudet Research Institute, 1996). While spoken English is clearly not accessible to these individuals, we must also conclude that in many cases English text is not accessible either. Thus many Deaf and Hard-of-Hearing teens and adults miss the opportunity to have confidential conversations with their doctors, lawyers, teachers, as well as assistance in government offices such as the post office and Internal Revenue Service. Assistive technology is needed that will improve real-time communication between Hearing and Deaf and/or Hard of hearing individuals. This device will not replace the sign language interpreter, but will offer the opportunity for privacy and a mechanism for improved communication when an interpreter is not available or independence is preferred.

SigningAvatar(tm) Technology

SigningAvatar(tm) technology uses computer generated three dimensional (3D) characters (also known as avatars) to communicate sign-enabled or scripted English text in variants of ASL. In this technology each sign is stored as motions of body joints as a function of time. Each facial expression is stored as motions of control points of the face. The use of animation data, as opposed to video formats, offers several key advantages over other graphical representations of sign currently available over the Internet, Intranets and on CD-ROM:
1. A simple Scripting Language tells the avatar what to sign. Recently, this script can be automatically created in real-time for Signed English and semi-automated for interpreting variants of ASL.
2. Whole sentences of signs can be linked together smoothly, without abrupt jumps or collisions between successive signs.
3. Manual signs and facial expressions can be combined in any desired manner.
4. Speed and repetition of signing content can be easily controlled by the author or user.
5. Signs animated for one character can be easily applied to other characters. These characters can include different ages and ethnic/gender appearances.
6. Lip-synched voice may also be provided along with Signed English, so the message is signed and spoken simultaneously in response to a text input.
SigningAvatar(tm) technology includes signs for over 7,500 words. This technology is already in use for improving learner comprehension. During the past year, over 30 school systems and resource centers across the United States have used a ground breaking educational CD-ROM featuring SigningAvatar(tm) signing characters.

Sign4Me(tm) Communicator

Description

Sign4Me(tm) Communicator is an automated English text to Signed English and lip-synched speech translation device. English text may be typed or pasted into the device, resulting in Signed English and voiced English outputs. Buttons for short cut inputs will also be available. An example use of short cut buttons is for an answer to a typical question in the post office, such as, "how much is a postage stamp for a letter?"

While this device will automatically translate the input, users will have visual identification of words with multiple senses and words spelled incorrectly. They may choose to use the controls provided to modify the message by choosing an alternative sense of a word and by adding facial expressions. This ability will allow variants of ASL, such as Pidgin Signed English (PSE), to be communicated as well as Signed English. Naturally, variants of ASL will require the user to possess sign language skills, whereas Signed English translation can be automatic. Voiced communication may be selected and may be used simultaneously with the signed translation. This is especially useful for hard of hearing individuals who may have some residual hearing or benefit from additional cues when communicating in Signed English.

Automated Generation of SigningAvatar(tm) Scripts

A scripting language tells the 3D characters what to sign and the appropriate facial expression to use as well as what to say when speech is selected. Until now, this script could only be authored by individuals, skilled in sign language and trained in the conventions of the SigningAvatar(tm) scripting language. This product introduces new capability that takes English text typed or pasted into a text field and processes it through SigningAvatar(tm) translating software and the natural language processing (NLP) software. The SigningAvatar( translating software and NLP software are tied together for immediate translation into Signed English. The NLP software assists this process by selecting the most appropriate sense for a multiple meaning word based on part of speech and frequency of use.

Demonstration

During this presentation, Sign4Me(tm) will be demonstrated using a PC laptop. The presenter will also ask the attendees to suggest sentences that they would like translated.

Conclusion

The Sign4Me(tm) Communicator will be available by June 2002. Beta testers from across the United States will evaluate this product prior to release. This assistive software will improve real-time communication between Hearing and Deaf and/or Hard of Hearing individuals. It will not replace the sign language interpreter, but will offer the opportunity for privacy and a mechanism for improved communication when an interpreter is not available or independence is preferred.

References

B. Holt, Judith A., Traxler, Carol B., and Allen, Thomas E., Interpreting the Scores: A User's Guide to the 9th Edition Stanford Achievement Test for Educators of Deaf and Hard of Hearing Students, Gallaudet Research Institute Technical Report 97-1, Gallaudet University, Washington, DC, 1997.

Gallaudet Research Institute, Stanford Achievement Test, 9th Edition, Form S, Norms Booklet for Deaf and Hard of Hearing Students, (Including Conversations of Raw Score to Scaled Score & Grade Equivalent and Age-based Percentile Ranks for Deaf and Hard-of-Hearing Students.) Gallaudet University, Washington DC, 1996.


Go to previous article 
Go to next article 
Return to 2002 Table of Contents 
Return to Table of Proceedings


Reprinted with author(s) permission. Author(s) retain copyright.