2000 Conference Proceedings

Go to previous article 
Go to next article 
Return to 2000 Table of Contents

Mouse Talk

Jamie Judd-Wall, Executive Director
Technology and Inclusion
P.O. Box 150878
Austin, TX 78715-0878
Phone: (512) 280-7235
FAX: (512) 291-1113
Email - jamie@taicenter.com
Website: www.taicenter.com

The ability to key letters and numbers and the ability to control the mouse are the essentials of computer use. For computer users with physical disabilities, this has usually meant the use of a scanning system. Letters numbers and/or mouse movements are offered on the monitor in a sequential format and the user activates a switch or series of switches to make a selection among the offered choices. It is an extremely tiring and time consuming process. Text generation is almost always less than 10 words a minute, even with prediction systems.

Current advances in on-screen technologies have made the scanning process slightly faster and easier. Options presented on the monitor reduce the requirement of shifting visual fields, but the scanning process remained essentially unchanged.

Direct selection remained largely unavailable to the computer user with severe physical challenges. Much of this has changed in recent years with the advent of newer technologies and on on-screen keyboards.

Our current project extends that application into the realm of voice recognition technology. In this project, computer users with severe physical disabilities and inarticulate speech (speech that is not recognized by an unfamiliar partner or a computer software system) train a closed set of utterances which are programmed to replace the traditional mouse movement commands in a voice recognition program. After a period of training and practice, the computer system replaces the original voice commands for mouse movements with the closed set. The computer user selects 'keys' on the 'virtual keyboard' on the computer monitor by moving the cursor with the newly trained voice commands. The voice recognition system is transparent, able to key into any application including internet systems. For the users in our project, ranging in age from 15 to 25, the experience has been extraordinary. We will use the actual software system and video tape samples to demonstrate the process to the audience.

In this paper we will review the statements programmed into the closed set and describe the training process. The Sound Commands voice profile contains over 50 commands which can be accessed with unique utterances of combined utterances. It is necessary for the user to be able to create a minimum of 5 different utterances. These will be translated into programmed utterances representing "UP", "DOWN", "LEFT", "RIGHT" and "CLICK" mouse movements. Additional unique utterances or combinations of utterances can be used to represent additional mouse movements such as "UP-LEFT" or"DRAG DOWN".

The Sound Commands vocabulary set is programmed into Dragon Dictate software. The specific utterances are then trained into the Sound Commands file which has been rogrammed into Dragon Dictate software. Training of the user's utterances follows. This can sometimes be a time consuming process. Individuals with inarticulate speech daily fluctuations in voice quality are common. In order for the voice system to work ffectively, recognition rates need to be trained at or above 95% recognition. The voice recognition system learns speech so thoroughly that one or two missed recognitions can lead to a string of errors being placed into the voice file. Missed recognitions are not easily corrected by individuals with the combination of severe physical disabilities and inarticulate speech. As the individual is trying to correct misrecognitions, he/she continues to dictate causing the system to belnd correct and incorrect recognitions together!

| Top |

The goal of initial training is to master the 5 basic commands. These commands are used to access an on-screen keyboard for word processing/text entry ... we use the REACH keyboard with SoothSayer word prediction. As the user builds their skills they are able to access pull down menus and even the URL line of various internet servers! The user moves the mouse to the URL address line, clicks and then moves the mouse to the on-screen keyboard. Frequently visited site accresses are pre-programmed into keys on the keyboard ... or the user can type the address using the onscreen keyboard. A click on the return key and they're crusin' the net ... no scanning required.

As the basic 5 commands re mastered, the user trains additional commands from the master set by using additional unique utterances or combinations of the 5 utterances. The same high level of recognition is required with each new command that is trained. Unfortunately it doesn't take much in the way of repeated missed recognitions to un-do months of dictation! Every missed recognition must be immediately trained or the error is integrated into the system.

We have found that the frequent repetition of the basic mouse commands has improved the articulation of some of our users, enabling them to access more of the Sound Command items with unique utterances. In addition we have been able to cross train technologies so that we can use some of the Sound Command utterances to access pages of a text document or listed items on a prediction list as well as the basic mouse functions of the Sound Command system.

| Top |


Dragon Dictate by Dragon Systems, 320 Nevada St., Newton, MA

Reach Keyboard, SoothSayer Word Prediction by Applied Humand Factors, Box

781076, San Antonio, TX

Sound Commands by MetroPlex Voice Computing, Box 121984, Arlington, TX

Go to previous article 
Go to next article 
Return to 2000 Table of Contents 
Return to Table of Proceedings

Reprinted with author(s) permission. Author(s) retain copyright.