1993 VR Conference Proceedings

Go to previous article 
Go to next article 
Go to Table of Contents for 1993 Virtual Reality Conference 

Modifying The Computer Interface To Meet The Special Needs Of The Disabled: The Sense Of Touch

A. David Johnson
TiNi Alloy Company
1621 Neptune Drive
San Leandro CA 94577
Tel. 510-483-9676
Fax 510-483-1309

Computers change peoples lives

We have an opportunity to change the world of the disabled. Steve Jobs is said to have challenged John Sculley to change the world by making the Apple personal computer accessible to everyone. I know someone whose life has been changed by the computer: Neil Marcus, who suffers from general dystonia. He cannot reliably move his limbs, and has great difficulty speaking. When I met him a decade ago, Neil was travelling around Berkeley in a motorized wheelchair (this mobility was the beginning of his freedom) and often came to watch what I was doing in my garage. He was also a frustrated writer who pecked out sentences on a typewriter. We tried to use the computer, Wordstar, without much success: striking two keys simultaneously was a great effort. But Neil did not give up: he got another word-processor and began to write. Soon we learned that his poems were being published.

Funny things happened. Neil had a sense of humor, an appreciation of the absurd. He wrote of himself as a break dancer. One day my telephone rang, and a message relay service was on line saying that I had a message from Neil Marcus. There was a long delay, and the operator laughed and said that the message was "my computer squeaks!". I visited his apartment and found that the computer did indeed squeak, for no apparent reason. Neil wrote anecdotes about amusing things which happened when one is in a wheelchair, and wove these vignettes into a series of skits. This became a play, in which Neil plays himself. Produced by Access Theater of Santa Barbara and shown around the country, it has received national recognition. Despite his severe disability, Neil has created a work of art. Neil's genius is to show us that we are all disabled, and it is O.K.

He could not have done it without the computer.

He also affected my life. Neil had special needs. His skeletal muscles function badly, but his eyes work perfectly. I visited Smith-Kettlewell with Neil to see whether Eric Sutter's methods of using eye movements to input information to the computer could help. This did not work very well for Neil. But while I was there, Sutter introduced me to the problem of Braille displays for blind computer users, and thus started me on the path which has brought me here today. We spent two years trying to solve the problem of full-page refreshable Braille. We designed a machine that has too many moving parts and is not practical to manufacture. [1]During this time I got acquainted with several inspiring blind people, and have not given up hope that some day there will be an inexpensive way for blind people to communicate with the computer. It will change lives.

VR and limitations to computer interfaces

The computer interface we know today, consisting of keyboard and visual screen with or without a mouse, is to be seen as an obstacle. Tom Furness, in congressional hearings on virtual reality (led by Al Gore) stated that the goal of VR is to remove this barrier between the intelligent person on one side of the interface, the speedy reliable information handler and processor on the other. We should be able to reach across this interface.

The tools of VR make it possible to do so. These tools can be employed by disabled people to overcome their obstacles to information access. Disabled people have special needs.

It is clear that ready access to an entire page or screen of information through a computer interface in the form of refreshable Braille would be a boon to the blind community. Efforts have been put forward by several research and development groups, but there is no inexpensive mechanical multiple-line Braille display on the market. [2]The role of sensory feedback in computer interfaces.

The sense of touch is taken for granted in everyday life, when we dress ourselves and when we manipulate tools of all sorts. But few people have thought seriously about giving the computer the power to communicate information by touch. This is tactile feedback.

Can the computer transmit useful information via the sense of touch? Suppose that as you move a cursor across a CRT screen you could feel in your fingertip the lines, shapes, and icons on the screen. It would help your to work faster, with less strain. There are many potential applications: CAD. Icons. Text. Graphics. Warnings. Alert to developing situations. Non-intrusive, secret communications. Tactile feedback might enable a person without sight to get some of the benefits of graphical user interfaces. Spreadsheets will become useful tools for blind users.

I believe that the advantages of tactile feedback will become apparent to all users of computer interfaces, but the visually disabled stand to benefit more than the rest of us. We are putting together hardware and software which should give blind computer users most of the benefits of a full-page Braille display. I believe that a product will result which has a cost low enough to be accessible to everyone, and will meet an identified need.

What we have designed is a new computer interface which combines several existing technologies with tactile and proprioceptive feedback.

The tactile Braille keyboard cursor-control interface

Some key observations:

Here is a description of the current effort to develop a tool useful for blind users.

The goal of this effort is a refreshable "Braille" system to make screen and page formatted information accessible to blind computer users. Mechanical tactile stimuli are presented to individual fingers, representing dots of Braille-coded characters, so that reading is the inverse of writing with a Braille keyboard. The keyboard, normally an input device, is enhanced to serve both input and output functions.

Pointing and page location information is provided by moving the Braille keyboard in x and y directions so that rows and columns of text are accessed utilizing the same arm motions employed in reading embossed paper Braille. Keyboard motion translates a cursor and the proprioceptive sense enables the user to sense movement from place to place over a virtual page. We will explore an alternate method in which a thumb-operated device is used to simulate motion within rows and columns.

A hardware design combining row and column scanning with eight-finger tactile stimulation are being integrated with a Braille keyboard for input and output of textual information to and from a host computer.

Several vendors provide single-line mechanical Braille systems.

[3]A single line of text does not provide ready access to information which is two dimensional, or page-oriented. The user must interrupt reading to move the window of text from line to line.

As an alternative solution, we provide a full screen of information by allowing the reader to scan a "virtual page" and update a refreshable display as each character of information is encountered. This gives the reader a means of reading text on a page while providing a sense of where useful information is found. Given the increasing complexity of computer information and the increasing use of spreadsheets, this is a critical capability which is lacking in existing devices.

A special kind of single-character display is used. A unique virtual refreshable Braille cell is moved by the user's hands so that rows and columns of text are accessed. It is not satisfactory to attach a six-dot refreshable character to a computer mouse for two reasons: the blind user has difficulty navigating the mouse on the screen, and also has difficulty reading the characters formed by multiple pins which move vertically under one fingertip. Both of these difficulties are circumvented by using only one dot under each fingertip and by providing a mechanical guide so that motion in the horizontal direction, along a line, is separate from vertical motion, up or down on the virtual page.

Two methods of navigation on the page are being tested. In the first method rows and columns of text are accessed in a way that utilizes the same arm motions employed in reading embossed paper Braille, and the proprioceptive sense enables the user to move from place to place on a virtual page of text. Layout is especially important in recovering meaning of text and relationships of data in spreadsheets. It has been shown that format-related information can be assimilated through a scanning procedure which involves arm motion. The proposed device takes advantage of this relationship.

In the second method a special version of a device called the "Isopoint" is positioned close to the keyboard so it can be operated with the user's thumbs. [5]Isopoint consists of a small horizontal cylinder which can be rotated to indicate y movements and slid along a rod to indicate x movements. Its advantages over a mouse or track ball are that it takes up less room, it decouples x from y so that operation by a blind person is easier, and it can easily be made to indicate absolute instead of relative position. The first method more nearly simulates the process of reading embossed Braille, while the second method will be less tiring to the arms and will be usable in portable devices.

Some manufacturers' systems furnish format information by providing joysticks, sliders, or pushbuttons as a means of navigating around the computer screen. These have the drawback of presenting information about page layout in a piecemeal fashion. Moving around a page by the repeated pressing of a button or by reaching away from the text to manipulate a lever is an unnatural operation which distracts the user from the task of reading text.

In very context-specific applications such as computer programming, spreadsheets, and processing of forms, the use of Braille provides advantages not available through voice synthesis systems. Sound systems do not have the accuracy or scanning speed that these applications demand. Furthermore, to access information quickly it is essential to have a sense of the location of data on the page.

Arm motion is a natural and direct movement used to locate position on a page of embossed Braille. Scanning a virtual tactile screen over a page of computer text imitates this process by bringing into play the proprioceptive senses. It has been demonstrated that providing a fixed track as a reference line for vertical movement of a 40-column display yields a greatly enhanced perception of layout.[6,7] Since most readers read Braille sequentially with one finger, reducing the virtual screen window to a single character may not greatly reduce reading speed or comprehension.

Cursor tracking and control is also a problem for blind users. Tactile feedback provides a possible solution. Vibration of the keyboard at high frequency can be used to identify when the cursor is under the virtual character: high frequency vibration (several hundred Hertz) selectively excites the Pacinian corpuscles and can be separately interpreted.[8] We plan to use this to convey non-textual information simultaneously with alphanumeric information. We will add an electromagnetic solenoid vibrator to the keyboard. This will be utilized to display cursor position: as the display slides over the cursor, a high-frequency signal will be received by the hands.

A single character with pins moving under the fingertip does not give the same information as a row of character sliding under the fingertip.[9] But the presence or absence of a single tactile stimulus on a fingertip is unambiguous: spatial resolution on an individual fingertip is not essential. Individual tactors can provide whatever intensity of stimulation is necessary. Communication of Braille information through separate fingertips may make textual information accessible to people with mild peripheral neuropathy. What we hope to learn from first prototypes.

Questions to be answered with the prototype device include these:

Some bigger questions also need answers:

Some concluding remarks:

A full-page display would fulfill one of the objectives of the Americans with Disabilities Act by making computer information, particularly spread-sheet calculations, more accessible to blind users.

The computer-literate blind community endorses a full-page Braille display. But previous attempts to make a full-page display resulted in devices too expensive or too complex. The proposed mechanism should provide most of the benefit of a true full-page display at a fraction of the cost.

It appears feasible to develop an elegant and simple means of implementing the X-Y scanning technique in such a way as to make it a reliable, easy-to-use, and easily manufacturable input-output device.

A caution I have learned from the blind community is this: we must always be careful not to solve the wrong problem. Some technology which looks promising to sighted persons is next to useless to a blind or otherwise disabled person.

At the same time, it is unfair to ask a disabled person what he would like to see provided. If one has not used a tool, it is difficult or impossible to predict the impact it might have.

We should move cautiously forward hand in hand with others who have different capabilities and skills, remembering that we are all disabled.



Go to previous article 
Go to next article 
Go to Table of Contents for 1993 Virtual Reality Conference 
Return to the Table of Proceedings 

Reprinted with author(s) permission. Author(s) retain copyright.