2000 Conference Proceedings

Go to previous article 
Go to next article 
Return to 2000 Table of Contents

Innovative Technologies for People Who Are Blind, Deaf-Blind, or Have Low Vision: Highlights of Projects from the Smith-Kettlewell Rehabilitation Engineering Research Center

Deborah Gilden

For about a quarter of a century the Rehabilitation Engineering Research Center (RERC) at The Smith-Kettlewell Eye Research Institute in San Francisco has been developing new devices for people with blindness, low vision, and deaf-blindness.  More recently the program has expanded to include research on developing new, more sensitive, practical tests of low vision.  This paper graphically demonstrates the wide variety of devices and activities engaged in at this research center.


The devices that come out of the Smith-Kettlewell RERC generally involve sensory substitution.  That is, information that is presented visually to sighted people is replaced with auditory and/or tactile representations of the same information.  The auditory information might be speech or non-speech sounds; the tactile information might be Braille or other raised symbol system.  Sometimes vibrotactile displays are used in devices for deaf-blind people.


The devices can relate to any life activity, e.g. vocation, education, recreation, orientation and mobility, activities of daily living (ADL), or health.


In an effort to (1.) use our resources efficiently, (2.) consider ultimate affordability by disabled consumers, and (3.) focus on research and development efforts unlikely to be conducted by for-profit companies, our projects have traditionally sought low-tech solutions.  As the cost of computer technology has drastically come down, however, we are developing more sophisticated devices.  Many of our devices incorporate speech chips, and a new project is in the futuristic area of artificial vision.

| Top | 

Low Tech for Low Vision:  How Low Can You Go?


Some people with low vision could read text if they held it extremely close to their face – except that their head blocks out the needed light.  The low tech solution was to add a miniature krypton light bulb in a tiny reflective cylinder to the spectacle frame.


A problem that is common even among those with normal vision – especially older people – is seeing the markings on home entertainment consoles such as VCRs.  These are often present low contrast buttons such as dark gray on black.  We have made these controls extreme easy to see as well as their functions easy to understand by simply by covering them with  very bright (“fluorescent”) tapes cut in simple descriptive shapes such a triangle for “fast forward,” a circle for “Stop,” etc.

| Top |

A Closer Look at Low Vision


Standard visual acuity testing provides little insight about the unique nature of the vision of individual’s whose sight is significantly reduced.  Knowing details about both the loss and the remaining vision is vital for developing maximally effective customized rehabilitation programs.  Researchers at Smith-Kettlewell have developed tests to reveal this crucial information.


The vision of many individuals degrades in conditions of dim lighting and/or glare.  Because in traditional vision testing the patient is asked to read black letters against a white background, it presents an inaccurate picture of daily visual functioning. 

| Top |

The SKILL Card 

The SKILL Card (Smith-Kettlewell Low Luminance, Low Contrast Card) is a low tech solution to bringing real-world conditions into the eye doctor’s office.  It simulates low lighting conditions by presenting black letters on a carefully calibrated and carefully rendered shade of gray background.  Reduced foveal sensitivity in low lighting is associated with certain eye diseases as well as with aging per se.

| Top |

The Glare Recovery Test 

Many every day environments introduce glare and rapidly changing light levels into the viewing scene.  This happens, for example, during nighttime driving when oncoming headlights come into view. The Smith-Kettlewell RERC has developed a test using a glare source, in combination with the SKILL Card, to determine a person’s ability to adjust to these changing conditions.  In teaming up with the Buck Center for Research on Aging, we tested a large population of older persons, and found that the time it takes to recover from glare increases dramatically with age.  Changes in lighting levels that take young people only a few seconds to adjust to may take an 85 year old longer than one minute.

| Top |


Macular Mapping Test


Many people’s low vision is the result of functional “holes” in different parts of their retinas.  Usually vision in the fovea is the first to deteriorate, followed by patches of visual reduction in the surrounding macula.  Knowing which retinal areas retain vision, and how sensitive each is, can help these patients devise a “looking strategy” to greatly improve their visual functioning.


A quick and inexpensive test has been developed at Smith-Kettlewell to give both the doctor and the patient a picture of how different parts of the macula are functioning.  The patient looks at the center of a pattern displayed on a computer screen and tries to recognize letters that are flashed on different parts of the screen.  Although the Macular Mapping Test takes only 3 minutes, it provides instant results.  Understanding these results has given many people with age related maculopathy and their families valuable insights about why it seems that they can and can’t see, as well as a tool to help them optimize use of remaining vision.

| Top |

Text Access for Blind Individuals


The Fax Reader Service


The Fax Reader Service is a unique approach to providing blind consumers a way of accessing print information.  They merely fax the material in question to a sighted reader -- or to a machine with optical character recognition -- and the person or machine reads it to them over the phone.  By using a flat bed scanner even the text on food boxes (e.g. product names and cooking instructions) may be faxed in this manner.


The Display Reader


We have developed a prototype device that can “read” light emitting diode (LED) displays and liquid crystal displays (LCD’s) and then present this information via a speech chip.  This has the potential to allow totally blind consumers access home appliances and entertainment systems, etc.

| Top |

Graphics Access for Blind Individuals


Full Page Braille Display


Ever since the first line of mechanical Braille was developed over two decades ago, there has been a dream for a full page of volatile dots.  Not only would this enhance reading text, but it would also provide a new medium for displaying graphics.  The physical space needed for the mechanism of each dot, however, along with each dot’s high cost, have precluded the realization of this dream.


Thanks to an entirely new concept in dot display technology, however, a full page of mechanical Braille may soon “be at hand.”  This innovative approach precludes the need for moving parts, giving the device the potential to be both reliable and affordable.  In fact, this approach could also be used for putting refreshable Braille labels on appliances so that users could read their displays.

| Top |



KnowWare(TM) is a virtual reality map based on a system developed by Artificial Reality Corporation (ARC) in Connecticut.  The Smith-Kettlewell RERC has been working with ARC on using the system to address graphical needs of blind consumers.  The map-reading application was the first effort toward this end.  It allows a blind user to move an outstretched finger around an invisible, non-tactile map, in order to hear location-based information.  Whether it’s the names of states, countries, continents, or bodies of water; information about the geology, meteorology, or politics of a given area; or any other relevant information, KnowWare(TM) can provide it instantly.  Any type of information and be programmed into the system at any level of detail, and in any language.

| Top |



Another approach to graphical access is Sonification Tools.  This technique was just developed at the Smith-Kettlewell RERC laboratories thanks to a blind University of California, Berkeley, graduate student.  This young man’s own need to know the shapes of graphs generated by a software application called MatLab(TM) spurred him to devise a solution.  He created a program that displays the visual graphs as analogous changes in auditory frequencies – a concept developed by the RERC in the 70’s for providing access to waveforms on oscilloscopes.  The new program includes sophisticated options for different types of auditory presentations, and different degrees of detail.

| Top |

Hear Here!  Technology for Knowing Where You Are


Talking Signs(TM), Talking Signs for Museums, and Talking ATM’s


The long white cane continues to be the basic mobility tool for blind people.  Knowing one’s location, however, requires more sophisticated technology.  Talking Signs(TM) developed by the RERC is gradually being installed in more and more municipal and university locations for this purpose.  Talking Signs(TM) consists of an infrared transmitter with a preprogrammed speech message, and a receiver with a speaker.  The transmitter is installed in places such as street corners, train stations, the fronts of public buildings, and near doors with room numbers.  When a blind pedestrian scans the area with a hand-held receiver, the infrared signal is detected and heard as a verbal message emitted by the receiver.  Because of the nature of the infrared light, the user can easily determine the exact directional location of the Talking Sign(TM).


The basic design of this device lends itself to many other applications.  For example, a blind bank customer can use the receiver to hear what is being displayed on the screen of automatic teller machines which are equipped with the appropriate transmitter.  Another variation is Multilingual Talking Signs for Museums.  This device is being developed with assistance from Mitsubishi Precision not only as a product for blind users, but also for the sighted public as well.  This device takes advantage of flash memory card technology.  It allows museum-goers to merely aim a receiver in the general direction of an exhibit in order to hear a large amounts of detailed verbal information about it.  The Museum Talking Signs are able to store such a large amount of information that patrons can turn a knob to choose any (or all) of three types of information for each exhibit.  For example, a possible combination might be:  (1.) descriptions designed specifically for blind persons of physical features of the display, (2.) detailed information about the exhibit appropriate for adults, and (3.) more elementary information about the exhibit to make it appropriate for children.

| Top |

Remote Sighted Guide

The blind pedestrian using Remote Sighted Guide technology is equipped with a camera that can send a picture of the local scene to a remote location, and a microphone and speaker that allow for two-way communication.  When information about the area is desired, the blind user scans the area with the camera and queries the “remote sighted guide” (a human in a remote location who can see the camera’s view) to describe what the scene.  This use of a human viewer renders this approach the most intelligent orientation device in the world.


Ideally, however, the intelligence would be built into a device.  The state-of-the-art of today’s computer vision is too premature for this, but we are taking initial steps toward this end.  The Display Reader described above marks the potential of a device that could eventually read any text on any sign.  Taking such technology to its ultimate level would result in a machine that could recognize objects and actually describe scenes.

| Top |

Communication and Telecommunication for Deaf-blind Individuals



Dexter, the Robotic Fingerspelling Hand, allows anyone to type a message to a deaf-blind person who can read fingerspelling.  The original concept and prototype of a mechanical hand for tactile fingerspelling came from the Southwest RERC in San Antonio.  After a long hiatus, the Smith-Kettlewell RERC revived the project.  Working first with Stanford and the VA Rehabilitation Research and Development Center in Palo Alto, and then with Upstart Robots in San Francisco, several functioning hands were fabricated and tested by deaf-blind consumers.  The most recent unit even allowed for telecommunication.  That is, from a remote location, an individual could type a message on a TDD (telecommunication device for the deaf) and send it to Dexter, which would display the message letter by letter to the deaf-blind consumer feeling the robot hand.

| Top |


Commercialization of Dexter still remains a dream for the future.  In the meantime another approach might appear on the market in the year 2,000 –especially since the Smith-Kettlewell RERC is developing it at the request of Blazie Engineering.  Known as the TeleLite it is, in effect, a combination of a VersaBraille and a TDD.  Since it is a sort of modernized TeleBraille, deaf-blind users must be Braille readers.


Medication Reminder for Deaf-Blind


The deaf-blind population tends to need more medication than the population in general.  Remembering to take medication is especially difficult for deaf-blind people, however, as they cannot track the visual cues of time passage nor use auditory alarms.  We decided that for some people the answer to this problem would be a Braille wristwatch-type device that could be set tactilely, with multiple alarms that could be set tactilely, and an alarm signal that is vibrotactile.  A colleague from the University of Tokai joined us in developing a prototype unit.  More work is planned to refine the prototype and to make it smaller.  Once this is accomplished we will ask deaf-blind consumers to evaluate it.


Go to previous article 
Go to next article 
Return to 2000 Table of Contents 
Return to Table of Proceedings

Reprinted with author(s) permission. Author(s) retain copyright.