2002 Conference Proceedings

Go to previous article 
Go to next article 
Return to 2002 Table of Contents

Bridging the Gap between Aspiration and Capability for Aphasic and Brain Injured People

Blenkhorn, Paul: Professor of Assistive Technology, UMIST
Hawes Paul: Director, Sensory Software International

Lead presenter: Paul Hawes


For the brain-injured individual, communication and computer access can be difficult to achieve, owing to the combination of physical difficulties and cognitive impairment.  The problems are further complicated by the fact that the patient may wish to have access to the same type of activity as before the trauma, but no longer has the capacity for written expressive language.

We will show how the use of a computer system allows a very flexible approach which can combine the use of symbols and words to control standard computer software, providing the possibility for spoken communication, writing, email and environmental control.

Between a rock and a hard place

For the cognitively able adult with a severe physical disability, the use of a computer system with adapted input may be slow and sometimes frustrating, but it delivers a lot of rewards.  With a suitably adapted system, the user can communicate in writing or speech, use the Internet, use environmental control systems and computer based entertainment.

The practical problems of access to a computer system are largely solved, and refinements to reduce the inherent frustration of systems such as switch access are constantly being developed.

However, for a significant group of users, these adaptations are not enough.  People who acquired a disability though injury or stroke are frequently unable to use these systems because their ability to handle written language has been affected.  In these cases, a picture based system is the obvious answer, but most such systems are written for children.  For someone who has lived an active life before disability, with a range of interests developed over the years, such a system is a significant step backwards.

The result is that, all too often, the user is offered a greatly simplified communication system, perhaps with quite inappropriate content.

Symbol based software

Symbol based software comes in two main types:  symbol based communication systems and symbol based word processors, designed as a teaching aid.

Symbol communicators, also called dynamic screen systems, such as Winspeak or Mind Express  allow the user to build a sentence by selecting pictures.  The sentence may then be spoken. 

In order to be effective, these systems require a substantial vocabulary to be provided when the system is first introduced, as it takes time to build the number of grids needed for the user to say anything substantial.    These are called “Pre-stored Vocabularies” in the UK and “Applications” in the USA.  Needless to say, most of them are very child oriented, although work is now in hand to produce more and better symbol communication grids for adults.

We should take a closer look at these systems, because in a moment we will see how they can be used for much more than face to face communication.  Here are three that illustrate different ways of using symbols for communication.

The Chailey Communication System:  3,500 words arranged in hierarchical grids in colour coded topics.  Each word takes three selections: for example the word “sock” would be found in the “footwear” section of the topic “clothes”.

Ingfield Dynamic Vocabularies: Four sets of topic based grids.  All are similar in layout and organisation, but the higher levels have more cells in each grid, and more vocabulary.  The lowest level has a few hundred words of core vocabulary, while the highest has about 1500 words.  Because a topic based system places all the vocabulary for making a sentence on a given topic on the same grid, it is very fast to use but results in a smaller vocabulary.

CallTalk This is a more advanced system, and combines the best of both ideas to give access to a full vocabulary by using a combination of topic grids and pop-up grids.  Like IDV, it has grids where the special vocabulary is stored for different topics.  However, instead of simple sentence starters like “can I have” the topic grids contain links that give access to a full range or pronouns, verbs and adjectives that may be needed in any topic.

Symbol word processors, like Writing with Symbols or Clicker Writer have an editing window, where text may be created by selecting from a grid that contains symbols.

As with communication systems, these are limited for adult use by the type of content normally offered.  Moreover, older users will want their computer to work using standard software, not programs written as a primary school teaching aid.

Another problem with programs of this type is that they do not give the user independent control of the computer.  Most are completely closed, and only write to their own window.  Those that allow text to be sent to other applications are limited in their control of the computer and their ability to manage the overall system.  This is for the very good reason that they are designed to be used, under supervision, in a classroom.  Teachers need to use systems that keep the activities of the user under some control!

Software to control the computer

For many years, software has been developed to allow computers to be controlled with a simple input device, such as a switch.  These programs are called on-screen keyboards; well known examples include E-Z Keys and WiViK.

These can be used with any software, and provide a high degree of control over the computer, allowing the user to control standard computer applications, move between windows and emulate both keyboard and mouse functions with a switch.

However, the ability of these programs to use symbol graphics to help the user to create language is at best limited and in many cases non-existent.

In other words, there was a choice between systems that are cognitively accessible, but do not provide independent control of the computer system, and systems that give that independent control, as long as you can read.

A new kind of on-screen keyboard

What we needed was an on-screen keyboard that could be used with everyday Windows software for writing, communication and Internet access, but which also  has the capability to display graphics in its cells. 

To go one stage further, we need to make that on-screen keyboard able to read the grids that already exist for other software, so that users have immediate access to them.  This then means that the user can create text by using the symbols that have been designed for face to face communication, and gain control over a wide range of activities.

HandsOff was the first on screen keyboard to combine full control of the Windows environment with the ability to use published pre-stored vocabularies and, as far as we know, is still the only one to do so.

PowerPoint show of symbol to text applications

Here are some examples of using symbol screens to operate a computer system in different ways:

  1. Face to face communication using linked grids of vocabulary to build a sentence.  This is identical to the dynamic screen communication systems used by children, except that the content is modified.
  2. Writing plain text using symbol grids to generate the words.  Exactly the same grids are used to select the vocabulary, but this time the target is a standard windows application, such as a word processor or email program.
  3. Fast text selection for users with receptive language.  This works well for people who can read fairly well, but who cannot spell.
  4. Handling menus with a special symbol grid.  Here, we have taken Outlook Express as an example.  All of the menu commands (apart from configuration options) have been included in the grid, with a suitable graphic symbol to identify the function.  When the user wishes to start composing the message, he uses a link to the main symbol index.
  5. Address Book The grid shows digital photographs of friends and family.  Selecting the cell can either type the person’s postal or email address.  A similar grid could be used as a phone book, if the user has a computer controlled phone (see environment control below).
  6. Environment control systems controlled by the computer, allow the user to operate a range of devices around the house, such as a music centre, lighting or an adapted telephone.

Hardware issues

These techniques may be used on any Windows computer, and the software is not expensive.  However, for most users, a rugged portable device with a long battery life is needed, possibly mounted on a wheelchair.

Until recently, this was a significant barrier, and most people would choose a specially built device to use as a communication aid, such as the Liberator or Dynavox.   However, the availability of tablet computers with built in touch screens, extra robustness and good battery life has radically altered the way in which communication aids are now implemented.

Go to previous article 
Go to next article 
Return to 2002 Table of Contents 
Return to Table of Proceedings

Reprinted with author(s) permission. Author(s) retain copyright.