2000 Conference Proceedings

Go to previous article 
Go to next article 
Return to 2000 Table of Contents

Scripts on an AAC system

Norman Alm, Department of Applied Computing, Dundee University, UK
Holger Neumann, IGEL GmbH, Bremen, Germany
Hans van Balkom, Instituut vor Dooven, Sint-Michielsgestel, TheNetherlands


Augmentative and Alternative Communication (AAC) devices tend to be slow to operate, and as a result conversations can be very difficult and frequently break down. This is especially the case when the conversation partner is unfamiliar with this method of communication. A way of improving the performance of AAC devices by using scripts is discussed. A prototype system to test this idea was constructed and a preliminary experiment performed with promising results. A practical AAC device which incorporates scripts was then developed, and is described.

Existing Augmentative and Alternative Communication (AAC) devices A basic form of AAC device consists of a word board or display panel on which a number of language items are arranged in a grid layout. The user communicates by pointing to the language item he or she wishes to communicate. The communication partner then reads the language item directly from the word board or display panel. With the advent of microcomputers and speech synthesizers, it became possible for these word boards to become dynamic and for the user to able to "speak" with a

synthetic voice. Even with these systems it takes a great deal of time and effort to produce each speech act. This is because the speech acts usually have to be constructed word by word. The result can be dysfunctional communication caused by long pauses and low communication rate.

A number of techniques have been developed to try to improve the efficiency of use of AAC devices, in order that users might be able to interact more effectively with their communication partners. Despite these advances, there remains a need for improved AAC techniques to enable users to participate in lively, fast moving dialogues with a wide range of conversation partners, including people whom they have never met before. One of the goals of AAC research is to improve the design of AAC devices in order to allow their users to communicate more effectively. An ideal AAC device would have the following characteristics: production of speech acts quickly with the minimum of physical effort, low cognitive load during use, and a minimal requirement for training.

| Top |

Scripts in AAC

This paper describes a new method of organizing prestored utterances to attempt to overcome the memory load problem. It is proposed that users' memory load can be reduced by making use of their existing long term memory to help them locate and select appropriate utterances from the

AAC system. Schank and Abelson [Reference 1] proposed a theory that people remember frequently encountered situations in structures in long term memory which they called "Scripts". A script captures the essence of a stereotypical situation, and allows people to make sense of what is happening in a particular situation, and to predict what will happen next. The AAC potential of the scripts concept as proposed by Schank and Abelson has been recognized for some time [Reference 2], and the project described here is part of an investigation into their application in AAC. Other research [Reference 3] has shown the potential that similar schema-based techniques offer to AAC. Preliminary experimentation has been encouraging and justifies further research on their use.

Scripts are most likely to be useful in transactional type conversations, such as speaking with sales staff, as these tend to be very structured, and predictable in content and flow. Many AAC users find this type of conversation particularly difficult as it very often involves them talking with strangers who are not familiar with their particular method of communicating. It is proposed that if pre-stored utterances are stored within script-type structures inside an AAC device, then a user could use their own memory to quickly access these device scripts and thereby find appropriate utterances. A suitable user interface for the device would be needed to realize this concept.

| Top |

Initial experiment

An initial experiment was devised [Reference 4] to investigate the potential of a script-based approach to transactional interactions in AAC, and a prototype AAC system was developed to facilitate this experiment. The aim was to ascertain whether or not a transactional interaction could be conducted using a script-based communication system. It was decided to simulate a particular transactional interaction which could be reasonably expected to follow a predictable

sequence of events, i.e. one which would be amenable to the script approach, in order to find out whether a computer-based script could enable a successful interaction to happen. The transaction chosen for the experiment was that of arranging the repair of a household appliance

over the telephone. A script was devised which contained all the stages of a typical telephone interaction in which someone calls a maintenance company to request that they come to repair a domestic appliance (in this case, a cooker (kitchen stove) was chosen as the example appliance). The caller was assumed to be a non-speaking person, trying to interact with the appliance repair person at the maintenance company. The script was devised to cope with all contingencies which might arise in the interaction with the receptionist, and no facilities were provided for the user to type new material.

Results of initial experiment

The system was tested to see if it could actually accomplish the task by having a non-disabled person use it. The test was to attempt to convey all the necessary information to forty volunteers playing the role of repair personnel at the other end of a telephone link. In every case the dialogue managed to complete the required task. One subject had some difficulty completing the task because of not understanding the synthetic speech. All of the conversations were timed, and the computer-mediated conversations were found to take on average approximately 50% longer than natural conversations which were staged to complete the same tasks. It was particularly encouraging that the experimenter could achieve a functional goal (requesting a domestic repair) using only a script-driven system with synthetic speech output when telephoning inexperienced subjects. This prompted the view that scripts could play a very valuable role during functional interaction within a general AAC system. It was therefore decided to pursue the

concept more generally by developing an experimental script-based AAC system to enable users to conduct functional interactions and general conversation in a number of everyday tasks and situations.

The realization of a practical device

To take this work further a collaborative project was started to incorporate scripts into a more practical device, involving IRV (a research institute in The Netherlands), and two commercial companies (Kompagne, The Netherlands and IGEL, Germany). The project was funded through the European TIDE programme. The user interface is made up of three main components: rapidly produced speech acts, the scripts and a unique text facility. The rapid speech act component contains high frequency utterances used in the opening and closing portions of a conversation and in giving feedback, and consists of groups of speech-act buttons. The Scripts component is used in the discussion phase of a conversation, and consists of a set of scripts with which the user can interact. The Unique utterance component is used when there are no appropriate pre-stored utterances available, and consists of a virtual keyboard, a word prediction mechanism and a notebook facility.

| Top |


To provide access to a set of scripts, a scene-based user interface was devised. The scene approach was taken in order to give users a pictorial indication of subject matter, and to assist users with varying levels of literacy skills. In this interface scripts are presented to the user as a sequence of cartoon-style scenes. The scenes give the user an indication of the subject matter and purpose of the script, and assists the user to assess whether the script is appropriate for current needs. Each scene is populated with realistic objects chosen to represent the conversation tasks that can be performed. The user thus receives a pictorial overview of the script, what happens in it, and what options are available. This assists the user to see quickly what the script will be able to do in the context of the current conversation. An example of a scene can be seen in Figure 1, which shows a scene within the doctor script.

Many users of AAC systems have literacy problems and use symbol-based communication systems. The pictorial interface was devised to be accessible to users with varying levels of literacy skills. Also research into picture recognition and memory structures has demonstrated

that groups of objects organized into realistic scenes corresponding to stereotypical situations better assist recognition and memory compared to groups of arbitrarily placed objects [Reference 5]. The scene-based interface using a realistic arrangement of objects within a scene was therefore chosen to facilitate recognition and remembering by the user and thus reduce the cognitive load required to locate suitable objects during a conversation. Several types of objects exist, some allow the user to switch between scenes and others to allow the user to speak useful utterances. Special objects allow the user to speak detailed information regarding times, dates and numbers.

As it would be impractical to provide scripts for every conceivable situation, it was decided to provide users with a limited number of scripts together with an authoring package with which they can develop their own custom scripts with help from their therapists.


A text preview and display box appears at the top of the user interface. The main interface area (bottom right) contains the scene image. The function buttons on the left side of the interface are, from top to bottom: "I'm listening" rapid speech act button; button to access the main rapid speech act interface; scene navigation backtrack to previous scenes button; scene navigation overview button; tool button to access the notepad and additional system control facilities.

Figure 1: The Script user interface showing a scene from the doctor's script

Access to all the scripts is provided through a "town plan" scene. To move from one script to another requires the users to navigate through the "town plan". In order to make it easy to locate the "town plan", a special control, the overview button, was provided. Quickly produced

all purpose phrases and a unique text facilities are also available to the user at all times. It is always possible that an unpredictable event will occur which will require deviation from the typical script and is therefore necessary for the system to offer flexibility, so that the user can respond to unforeseen events and maintain the flow of conversation.

Development of scripts

After consultation with a group of AAC users (described in Reference 6) it was decided to develop a set of seven scripts. These scripts were chosen because they represented situations that users thought important, but found great difficulty communicating in. The seven scripts were: "Activities of daily living", "Restaurant", "Doctor", "Shopping", "Meeting someone new", "Talking about emotions" and "On the telephone". The scripts were developed by considering all the conversation tasks or goals a user would wish to achieve in any given situation. These conversation tasks were further broken down until each sub-task could be performed with a single phrase. A suitable phrase was then composed for each of these conversation sub-tasks. These phrases were grouped into sequences and scenes, and then mapped onto suitable props. The

complexity of each scene was carefully controlled to make the scenes easy to use, and easy to learn to use. Wherever possible the number of props within a scene was restricted to nine, as this is approximately the limit of human short-term memory [Reference 7].

| Top |


Potential users all reported liking the look of the interface an feeling that might help them to communicate better. An actual evaluation of this system showed an increase in total number of words produced and the words per minute and words per turn rate by the participant when using

ScripTalker in the role-plays compared to using their existing communication device.

Adapting the scripts and making new ones

With a separate program all scripts can be adapted to the user's needs. It is easy to alter utterances and select symbols. In this way the system can grow with the user's communication skills. At the moment there is a "ScriptAuthor" system under development. This is a program

added to ScripTalker, which will allow for very easy creation of new scripts as well as the individual adaptation of existing scripts to the user's needs. This enables the user or carers to include individual objects like photographs. ScriptAuthor will especially address the problems people have in defining a script. This problem is similar to the one many people have when they want to fill a communication device with messages: what comes where, how to formulate a phrase so that it is widely applicable, what graphics to use. It will also help them analyze

the various speech acts which need to be included in a script. Additionally, ScriptAuthor will allow for import and export of scripts, so that ScripTalker users can profit from scripts made for others.


ScripTalker was first marketed in Dutch and German versions. It will be available soon also for Swedish, Finnish, and English. The adaptation to foreign languages is very easy to do, by translating the prestored messages and utterances and changing the speech synthesizer. ScripTalker has been developed in such a way that all language-specific content is stored in separate files which can be edited by means of any ASCII editor.

| Top |


The incorporation of scripts into AAC devices has potential for helping non-speaking disabled people take an effective part in transactional conversations. The success of this type of AAC device will depend largely on the number and quality of the scripts that are available. In order to solve the problem of only having a limited number of scripts a

new piece of software is being developed which will allow speech and language therapists to easily create new scripts and to collaborate with other therapists through the sharing of scripts.


The development of ScripTalker was supported by the TIDE Programme of the Commission of the European Union under the ALADIN project. The project partners were The University of Dundee (UK), Instituut vor Revalidatievraagstukken (The Netherlands), Kompagne BV (The Netherlands), and IGEL GmbH (Germany). Further work in this field is being conducted under the REACT project, involving these four partners and Instituut voor Doven (The Netherlands).

| Top |


[1] Schank, R. and Abelson, R. (1977) Scripts, Plans, Goals, and Understanding, New Jersey: Lawrence Erlbaum.

[2] Kraat, A. (1987) Speech Therapist, Morton Roberts Center for Augmentative Communication, Queens College Speech and Hearing Center, Flushing, New York, USA, Private Communication, August.

[3] Vanderheyden, P.B., Demasco, P.W., McCoy, K.F. and Pennington, C.A. (1992) A preliminary study into schema-based access and organization of re-usable text in AAC, Proceedings of the RESNA '96 Conference, Salt Lake City, Utah, USA, 17th.-12th. June 1996, pp.59-61.

[4] Alm, N., Morrison, A. and Arnott, J.L. (1995) A communication system based on scripts, plans and goals for enabling non-speaking people to conduct telephone conversations, Proceedings of IEEE Conference on Systems, Man & Cybernetics, Vancouver, Canada, 22-25 October, pp. 2408-2412

[5] Mandler, J.M. (1984) Stories, Scripts and Scenes: Aspects of Schema Theory. New Jersey: Lawrence Erlbaum.

[6] Alm, N., Dye, R. and Harper, G. (1995) ALADIN - Advanced Language Device for Interaction", Proceedings of ECART 3 Conference, Lisbon, Portugal, 10-13 October, pp. 150-151.

[7] Miller, G.A. (1956) The magic number seven, plus or minus two: some limits on our capacity to process information, Psychological Review, 63, 81-97.

Go to previous article 
Go to next article 
Return to 2000 Table of Contents 
Return to Table of Proceedings

Reprinted with author(s) permission. Author(s) retain copyright.