2004 Conference Proceedings

Go to previous article 
Go to next article 
Return to 2004 Table of Contents 


THE COMMUNICATION ASSISTANT - CAPTIONING IN COMMUNITY VENUES

Presenters
Leanne West
Georgia Tech Research Institute
Electro-Optics, Environment, and Materials Laboratory
925 Dalney Street
Atlanta, GA 30332
Phone: 404-385-0405
Fax: 404-894-6285
Email: leanne.west@gtri.gatech.edu

Jay Sexton
Georgia Tech Research Institute
Information Technology and Telecommunications Laboratory
GCATT Building, 250 14th Street NW
Atlanta, GA 30318
Phone: 404-894-8191
Fax: 404-894-6285
Email: jay.sexton@gtri.gatech.edu

John Stewart
Georgia Tech Research Institute
Electro-Optics, Environment, and Materials Laboratory
IPST Engineering Center, 575 14th Street
Atlanta, GA 30332
Phone: 404-894-3459
Fax: 404-894-6285
Email: john.stewart@gtri.gatech.edu

Jack Wood
Georgia Tech Research Institute
Electro-Optics, Environment, and Materials Laboratory
Atlanta, GA 30332
Phone: 404-385-0405
Fax: 404-894-6285
Email: jack.wood@gtri.gatech.edu

The Communication Assistant is a wearable captioning system for use in a wide variety of public venues including movie theaters, museums, live theaters, schools, sports arenas, transit stations, and places of worship. This assistive technology system will allow users to easily receive information that is being presented audibly to the general public.

Researchers at the Georgia Tech Research Institute (GTRI) developed this technology to fill the void of accessibility in public places for people who are deaf or hard of hearing. The lack of captioning in community venues prevents equal access to information for individuals with hearing impairments and prohibits their full community involvement. With the Communication Assistant, venues will now have the opportunity to provide the same level of service to both their deaf and hearing patrons.

The Communication Assistant relies on mobile wireless technology and consists of three components: a transmitter, a receiver, and a display. Multiple system configurations are available. The venue and user preference determine the best configuration for any given situation. The venue management owns and operates the transmitter, including any equipment used to generate the captions. Users then borrow or rent the receiver and display from the venue, or they provide their own. The Communication Assistant is designed to work automatically in any location and uses the 802.11b wireless protocol. Therefore, wireless-enabled PDA's (Personal Digital Assistants) and laptops can be used as both receivers and displays. For many applications, though, users may require a display that is smaller, more private, and hands-free. For those instances, a miniature display manufactured by the MicroOptical Corporation is worn in front of one eye. This display is plugged into a PDA, which still functions as the wireless receiver and can be slipped into a pocket. The MicroOptical display can be attached to the user's glasses, worn on a small headset, or even permanently built into the user's glasses. Although positioned close to the eye, the display uses optics that make its screen appear to float several feet away, giving users relaxed viewing of both the captions and of the world around them. The display creates the illusion of captioned text overlaid on the user's visual field.

Input for the transmitter can come from three possible sources. In cases where captions can be created ahead of time, they will be stored digitally then transferred to the transmitter. A movie theater is an example of this situation. In all other cases, the captions will have to be generated in real time. One method of creating real time captions is by using a CART (Communication Access Realtime Translation) typist. A CART typist transcribes a speaker's comments verbatim using a stenograph keyboard and specialized software. The output from the stenograph station is delivered directly into the Communication Assistant transmitter. This type of captioning can be used at sporting events, live theater, business meetings, seminars, etc. The third source for captions is speech recognition technology. In cases where the speaker will be interacting repeatedly with individuals who are deaf of hard of hearing, such as in a classroom or business setting, the speaker can pre-train speech recognition software to his or her voice, speak into a high-fidelity microphone and deliver an accurate transcript to the transmitter.

In complex environments such as schools and movie theaters where there is the need for multiple captioning data streams, the wireless system will utilize multiple transmission channels. Channel options give the advantage of using the same hardware to access multiple streams of information in a single facility. Student users in an English class will tune to channel 1 for the transcript of their English teacher while users next door in history class will use channel 2. Channel options are just one form of system control given to users. The user interface will also provide options for text size, text and background colors, pop-up or scrolling captions, and the number of lines of text to appear on their display.

The National Center for Accessible Media, which provides captioning for first run movies and developed the Rear Window captioning system, has granted the project a temporary licensing agreement so that GTRI may design the Communication Assistant to be compatible with their data format. The organization has also given the project much support in the form of advice regarding user and venue requirements.

Proof-of-concept prototypes have been built and tested within the deaf and hard of hearing communities. A prototype ready for commercialization is currently being finalized. Initial testing occurred in Jan/Feb 2003. The Georgia Council for the Hearing Impaired (GaCHI) played a vital role in recruiting volunteers. In June 2003, the device was further tested at the Self Help for the Hard of Hearing (SHHH) Conference in Atlanta, GA. The following paragraphs are the results of the testing.

Including both rounds of testing, 63 participants were recruited to test the device, representing a variety of age groups ranging from 15 to over 75 years old. Volunteers included people who are hearing (usually an ASL interpreter or an interested family member) (11%), hard of hearing (71%), deaf from birth (8%), and late deafened (10%). 90% of the adult volunteers had a college degree or higher.

The tests were conducted in a simulated movie-theater setting with one to four volunteers at a time. After a brief introduction to the device, volunteers were asked to watch anywhere from 15 minutes of a movie to a full-length feature (up to 1 1/2 hours), while using the device to receive the captioning of the movie. The miniature displays clipped to eyeglasses were used to present the captions. The following information and quotes come directly from the questionnaires that the volunteers filled out after using the device to watch the movie.

Most volunteers thought the display "took some getting used to," but 65% of users said it took less than 10 minutes to get the display in comfortable reading position. 65% also said the text was easy to read. The most noted suggestions to improve the readability were to include a text-size option and focus-ability, both of which were not part of the original prototypes but are currently being included.

84% of the volunteers said the Communication Assistant would not make them feel self-conscious to wear or use. The few that said it would gave comments such as: "wouldn't mind in theater, but in lighted situation would be hesitant unless with others wearing it too."; "I am only self conscious a little. I would use it in a movie theater no problem. I might be more self conscious when lights are on."; "Only if I'm the only one using it."; "If it helps enough for me to get what is going on in the movie then I don't care what others think." 70% said the display fit well, but some noted that they were not used to wearing glasses or that it pulled to one side. 62% had no discomfort, but others noted some dizziness, headache, and/or eyestrain.

When comparing the Communication Assistant to other versions of captioning, 81% of people who had used open captioning still prefer it at the movies. Surprisingly for the researchers, for those that use sign language interpreters, most thought the Communication Assistant was about the same as or better than having an interpreter. "Even in lip reading I miss some words, therefore it's good to have a back-up. For the back-up, I would prefer to have the CA than an interpreter - I feel more independent."

General comments included, "This is a great product & I look forward to seeing it out in the field." "This is neat! A good device to carry around if no captioning available or if interpreter doesn't show up or not arranged. Ideal for young people." "Truly amazing." "A very worthwhile project."

Funding for further development in voice recognition technology for the classroom setting has been applied for. Additionally, an interactive model of the Communication Assistant that will allow a deaf user to ask questions or respond in other ways is being developed. With the addition of a mini-QWERTY keyboard and a voice synthesizer, deaf users will be able to communicate their questions and comments in places like classrooms and meeting rooms. The mini-QWERTY keyboard, as implemented on the RIM Blackberry and other text pagers and PDA's, is already commonly used in the Atlanta deaf community and across the country.

Funding for the Communication Assistant is provided by the National Institute on Disability and Rehabilitation Research of the U.S. Department of Education under grant number H133E010804 and the Georgia Tech Research Institute. The opinions contained in this publication are those of the grantee and do not necessarily reflect those of the U.S. Department of Education or the Georgia Tech Research Institute.

The development team would also like to acknowledge The National Center for Accessible Media and Thad Starner of the Georgia Institute of Technology for their input and recommendations.


Go to previous article 
Go to next article 
Return to 2004 Table of Contents 
Return to Table of Proceedings


Reprinted with author(s) permission. Author(s) retain copyright.