1999 Conference Proceedings

Go to previous article 
Go to next article 
Return to 1999 Conference Table of Contents


MULTI-MODAL INFORMATION PRESENTATION FOR COMPUTER USERS WITH DYSLEXIA, READING DISABILITIES OR VISUAL IMPAIRMENT

Peter Ridge
Architecture & Design
pridge@mindmaker.com

Tim Musgrove
Natural Language Technologies
tmusgrove@mindmaker.com

Jeff Savage
Intelligent Personal Assistants
jsavage@mindmaker.com

Mindmaker, Inc.
224 Airport Pkwy. #550
San Jose, CA 95110
(408) 467-9200
http://www.mindmaker.com

Overview

Today, the Internet has become a part of our personal and professional life. Accessing the wealth of text information on the World Wide Web is a must, but not everyone is able to realize the full potential of computer systems or the Web because reading or comprehension dysfunctions make using documents on a computer difficult. Moreover, for those who suffer from visual impairments like acute myopia, simply seeing the information can present a significant challenge. In order to increase the viewing enjoyment and understanding of text-based information for these users, a multi-modal software system that can present text in an audible and easily readable form with simultaneous associated graphics and sound is needed.

TextAssist

Millions of users already know the former version of TextAssist, developed by Creative. Mindmaker developed a new version of TextAssist that is bundled with Creative audio boards and is also available for use with all Windows-compatible sound devices. It is based on Lernout & Hauspie Speech Product's TruVoice text-to-speech synthesis technology and can read aloud text from any Windows application. By means of familiar CD player-like controls, the user can "play" text from applications such as word processors, e-mail programs and web browsers.

TextAssist is fully customizable, offering several different voices that can be modified in pitch, volume, and speed. Individual voice characteristics can also be tailored to the user's taste and associated with specific applications. For example, a voice with a bright and peppy feel can be associated with a word processing application while a serious and conservative voice could speak when reading spreadsheets.

TextAssist DX

Mindmaker not only improved the basic functionality of TextAssist since its conception as a text reader, but has developed a multi-modal version, TextAssist DX, as a specialized tool for the assistive market.

TextAssist DX toolbar
TextAssist DX toolbar

TextAssist DX window
TextAssist DX window

TextAssist DX extends the base product with the following features:

Enlarged, synchronized text highlighting

The highlighting feature displays a three window dialog to the user while text is being read. The upper window displays the text being read in an enlarged font to increase legibility. The size and color of the font and window background are configurable to the needs of the user. As the text is read, each word is synchronously highlighted to indicate the current word being read within the body of the text. This feature makes it much easier to follow the text being read.

In addition to the enlarged font, another window displays a smaller font view of the text. This "zoomed out" view gives the user a sense of where they are within the body of the document.

Associated images and sound

In a third window, images and animation are displayed as the text is being read. These images are triggered by the content of the document. For example, a sentence that describes an eagle triggers an image of an eagle as a graphic reinforcement of the word eagle. In addition to the image, sounds are also triggered as further enhancement to the content being read. In this example, the screeching of an eagle can be heard in the background.

Additional applications of the technology

In addition to aiding users with reading, comprehension or visual disabilities, TextAssist DX is also applicable to other areas. Children who are learning to read are benefited by the synchronized highlighting so that they know what words sound and look like. Also, the meaning of words that are unfamiliar to them are easily inferred by the associated graphics and audio effects. The same is true for adults who are learning a second language. Although TextAssist currently speaks English, Mindmaker will be adding support for other languages to bring the technology to more users.

The multimedia presentation capability of TextAssist DX takes the "books on tape" concept to the next level. Not only does the user hear the story, but animation, background sounds and sound effects add to the environment. With the addition of three dimensional sound generation technologies, the user can be fully immersed into the story.

Conclusion

Computer users with dyslexia, reading disabilities or visual impairment can get more out of text-based information by presenting it via multiple media channels. TextAssist DX displays text with a magnified font size, sequentially highlights the text as it reads it out loud, displays images that correspond to the context of the highlighted text, and generates pertinent sound effects. This simultaneous reinforcement effectively enhances the user's comprehension and enjoyment of the information.


Go to previous article 
Go to next article 
Return to 1999 Conference Table of Contents 
Return to Table of Proceedings


Reprinted with author(s) permission. Author(s) retain copyright.