2001 Conference Proceedings

Go to previous article 
Go to next article 
Return to 2001 Table of Contents


AAC Clinical Summary Measures For Characterizing Performance

Katya Hill, M.A., CCC-SLP
Edinboro University of Pennsylvania

Barry Romich, P.E.
Prentke Romich Company

Abstract

Augmentative and alternative communication (AAC) evidence-based practice requires quantitative measurement of communication performance. Methods and tools have been developed for collecting and analyzing data from the communication of people who rely on AAC. Data logging tools are available in various forms either built into modern high performance AAC systems or for use with older systems that have a serial port representation of language activity. Data collected using these tools has been analyzed using various computer software programs and manual methods that can generate information of interest to a spectrum of interested stakeholders. This paper identifies a set of summary measures most useful to clinical service providers and consumers, describes the value of each measure, and describes current methods of obtaining it.

Background

A performance-based understanding of communicative competence has long been a basic aim of the field of AAC. While the nature of communication includes many complex issues, recent developments in the area of automated language activity monitoring (LAM) have opened a new frontier allowing the clinical practice of AAC to be based on quantitative data. Language activity monitoring is the recording of the time and content of language events generated using the AAC system. While this does not represent total communication, for people who use high performance AAC systems it can offer the first quantitative measurement of at least a component of their communication performance.

Empirical research identifying the performance of augmented communicators has been limited at best. The few research studies that have investigated the performance of augmented communicators on AAC systems are confined to identifying vocabulary use with alphabet-based systems (Beukelman, et.al, 1984) or studying the efficacy of various rate-enhancement strategies (Koester & Levine, 1994). However, studies are beginning to appear regarding the performance of augmented communicators using the LAM to collect the data. These early studies based on LAM data have looked at the frequency distribution for the various language representation methods used by augmented communicators to generate spontaneous novel utterances during conversation (Hill & Romich, 1999a,b).

Gathering performance documentation on augmented communicators is essential to a thorough understanding of communicative competence with AAC systems. Such documentation provides "milestones of possibilities" to assess growth and evaluate AAC strategies and systems, thus serving as a basis for clinical decisions, outcomes measurement, and moving toward a quantitative definition of AAC communicative competence. Today AAC users and their facilitators must weigh the relative benefits and costs of available strategies on personal experience and intuition without much assistance from a research base (Beukelman & Mirenda, 1998).

AAC evidence-based practice requires quantitative measurement of communication performance. This paper presents a set of summary measures together with tools and methods to assist AAC practitioners in the clinical decision-making process. These summary measures provide the quantitative data necessary for a structured and methodical approach to measure performance and clinical effectiveness. When combined with the information gathered using traditional qualitative methods, AAC users and their facilitators will have what they need to weigh the benefits of available technology solutions and strategies.

Language Sample Collection, Uploading, and Editing

The LAM function records the content and time of language events (one or more letters or words). The LAM function is available as a built-in feature in modern high performance AAC systems, as an add-on for older systems, and as a computer software program. AAC language samples can be collected in clinical settings and in the natural environment. Each has particular advantages. Natural environment sampling over a long term is more indicative of actual use of the AAC system. However, the controlled structure of sampling in the clinical environment can provide information not otherwise available. This can include data that can be compared to normative data and to data collected previously from the same individual.

Language sampling procedures have been designed as part of a Small Business Innovation Research (SBIR) project on the LAM (Romich & Hill, 2000a). The procedures were developed with clinician and consumer input on selecting tasks for interactive communication. To date, clinical language sample collection has been based on one or more of the following procedures: picture description, interview, conversation, or clinician choice. The procedures adhere to traditional approaches to harvesting the most representative sample or largest sample of continuous spontaneous interaction from an individual as possible.

The LAM data can be uploaded into a computer using LAMupload or any properly configured serial communication program. Depending on the program being used to generate the summary measures, editing and coding may be necessary prior to entering the data into the analysis program.

Summary Measures and Analysis

The process of analyzing LAM data to generate summary measures and related reports can be implemented using various computer programs or manual methods. The initial identification of summary measures and analysis programs was based on survey data from AAC practitioners and consumers as well as collaboration with software developers. Presently no single program is available that can provide all measures. However, the LAM log file will need to be transcribed following the entry conventions and codes of the language analysis program. Hill has developed a set of transcription codes specific to AAC language activity (Hill, 2000). The description of the various AAC conventions and codes to facilitate the analysis of LAM data is beyond the scope of this paper.

The following list identifies the summary measures considered valuable for AAC clinical practice together with the available methods of deriving the specific summary measure. (S)ALT is Systematic Analysis of Language Transcripts (Miller & Chapman, 1991). (A)CQUA is Augmentative Communication Quantitative Analysis (Lesher, et.al., 2000). (M)ANUAL indicates that no program presently is available to generate this measure.

  1. Total main body words S, A
  2. Number of different word roots S
  3. Number of complete utterances S,
  4. Number of incomplete utterances S
  5. Number of total utterances A
  6. Number of spontaneous utterances M
  7. Number of pre-stored utterances M
  8. Mean length of utterance in words (MLUw) S, A
  9. Mean length of utterance in morphemes (MLUm) S
  10. Brown's Stage S
  11. Expected Age Range (months) S
  12. Average communication rate A
  13. Peak communication rate A
  14. Selection rate M
  15. Language representation method usage (LRM) M
  16. Number of errors by type S
  17. Non-Language Events M
  18. Related reports
    1. Raw LAM data NA
    2. Edited utterances M
    3. Coded utterances M
    4. Word list in alphabetical order S
    5. Word list in frequency order S
    6. Word list by LRM M
    7. Word list comparison to reference lists M

A. Total main body words is a measure of total communication. This measure is made following editing that includes coding of words selected in error and those selected in the process of generating a final word, such as may happen when using a word prediction system, for example. The error words are not included in the total.

B. Number of different word roots is a measure of vocabulary diversity. Transcript preparation is required by adding the conventions to indicate bound morphemes. This summary measure should only be calculated for AAC systems that provide for the individual selection of morphemes.

C. Number of complete utterances is the number of complete or whole sentences in the sample. This could include single word responses.

D. Number of incomplete utterances is the number of abandoned or uncompleted sentences in the sample.

E. Number of total utterances is the number of complete and incomplete sentences that have been identified.

F. Number of spontaneous utterances is the number of sentences that were generated through the use of individual words and phrases that have general application.

G. Number of pre-stored utterances is the number of sentences that were not generated using individual words and phrases that have general application.

H. Mean length of utterance in words is the average number of words in the sentences.

I. Mean length of utterance in morphemes is the average number of morphemes in the sentences. A morpheme is a unit of meaning. Some words have more than one morpheme.

J. Brown's stage is a language use rating based on normative data.

K. Expected age range is the number of months of age at which this level of language use would occur under normal development.

L. Average communication rate is the average number of words per minute, weighted by utterance length, found in the utterances (Romich & Hill, 2000b).

M. Peak communication rate is the largest number of words per minute found in an utterance longer than the mean length of utterance in words (MLUw).

N. Selection rate is the number of bits per second representing the speed with which the individual enters information into the AAC system (Romich & Hill, 2000c).

O. Language representation method use analysis measures the percentage of words generated using the primary methods of single meaning pictures, alphabet-based methods (typically spelling and word prediction), and semantic compaction.

P. Number of errors by type is a list of errors that can be attributed to various causes.

Q. Non-language events is a list of the non-language activity that occurred during the sampling procedures, such as turning the volume up or down, or turning speech on or off.

Of the above identified summary measures, communication rate, language representation method use, and frequency order word lists are among the most valued by AAC clinicians and consumers. In addition to the summary measures indicated above, other related reports provide value to the AAC clinical intervention and outcomes measurement processes. They include the raw LAM data, edited and coded utterance lists, and word lists organized by alphabetical order, frequency order, by language representation method, and by comparison to reference lists.

Report format

The information that is being used presently is shown, complete with data typical of individuals who rely on devices that support the three AAC language representation methods. Due to the constraints of this presentation, the formatting and a pie chart have been removed. A formatted version is part of the Procedure Manual for LAM, which can be downloaded from the AAC Research and Resources area of the Prentke Romich Company web site www.prentrom.com. Other related reports would be appended to this summary measure report.

--- LAM REPORT ---

Name: John Doe Date of Report: 000306

Location: EUP DOB:

Device: Pathfinder CA:

Language Representation Methods: Unity

Examiner: Melissa Length of Sample-total time: 20 min

Transcriber: Brad & Sara Length of Sample-total utterances: 17

Language Sample Context: Please check Briefly describe Activity/task

Dyadic Conversation

Conversation (2+ partners)

Narrative

Picture Description X Cookie Theft

Other: ________________________

TRANSCRIPT SUMMARY

SUMMARY MEASURES

RESULT

MLU in Words 9.29

SNUG Ratio

MLU in Morphemes 10.71

Total Pre-stored Messages 0

Brown's Stage

Total Spontaneous Utterances 17

Expected Age Range (mos.)

Type Token Ratio (TTR) .60

No. Diff. Word Roots 95

Total Main Body Words 158

Communication Rate

Selection Technique Rate

Non-Language Events

Language Representation Method Number of Words Percent

Total word count 158 100

Total Minspeak words 141 89

Total single-meaning pictures 0 0

Total spelled & predicted words 17 11

Total spelled words 7 4.4

Total predicted words 10 6.3

Predicted/ (spelled+predicted) 10/7 70

Total letter codes 0 0

Attached Summary Reports (please check)

Core/Extended Vocabulary Word List

Communication Rate

Selection Technique Rate

List of Spelled & Predicted Words

Types of Errors

Other

Language sample library

A language sample library has been established at Edinboro University of Pennsylvania. The individuals from whom the samples were collected are characterized to allow the review of samples that could represent reasonable expectations of future performance of others who rely on AAC. This library is expected to be available on a web site in the near future. Contributions to the library are welcome.

Discussion

Our understanding and evaluation of AAC communicative competence should be based on the foundation of quantitative summary measures. The use of AAC performance measurement tools can provide clinicians and consumers alike with quantitative information that can improve therapy and communication effectiveness. A scientific and methodical approach to the application of these new resources includes the identification and use of a standardized set of summary measures. Current and future efforts are focused on continuing to automate the analysis of LAM data to generate these measures.

Acknowledgement

The authors gratefully acknowledge the support of the National Institute for Deafness and Other Communication Disorders of NIH. An NIH Small Business Innovation Research (SBIR) grant provided funding for the early stages of this work.

Several individuals who rely on AAC have contributed to various phases of this work. They also have contributed language samples that have been the basis for current progress. In particular, the authors thank Snoopi Botten, Rick Creech, and Paul Pecunas.

References

Beukelman, D.R., & Mirenda, P. (1998). Augmentative and Alternative Communication Management of Severe Communication Disorders in Children and Adults. Second Ed. Baltimore: Paul H. Brookes Publishing Co.

Beukelman, D.R., Yorkston, K.M., Poblete, M., & Naranjo, C. (1984). Frequency of word occurrence in communication samples produced by adult communication aid users. Journal of Speech & Hearing Disorders. 49: p. 360-367.

Hill, K. (2000). AAC performance monitoring: Tools for quantitative research. ISAAC Research Symposium. Washington, D.C.

Hill, K.J., & Romich, B.A. (1999a). Identifying AAC language representation methods used by persons with ALS. American Speech-Language-Hearing (ASHA) Convention. San Francisco, CA.

Hill, K.J., & Romich, B.A. (1999b). Language activity monitoring for school-aged children: Improving AAC intervention. American Speech-Language-Hearing (ASHA) Convention. San Francisco, CA.

Koester, H.H., & Levine, S.P. (1994). Modeling the speed of text entry with a word prediction interface. IEEE Transactions on Rehabilitation Engineering. 2(3): p. 177-187.

Lesher, G., Moulton, B.J., Rinkus, G., & Higginbotham, D.J. (2000). A universal logging format for augmentative communication. CSUN. Los Angeles, CA.

Miller, J., & Chapman, R. (1991). SALT: A computer program for the Systematic Analysis of Language Transcripts. University of Wisconsin: Madison, WI.

Romich, B., & Hill, K. (2000a). Feasibility Study on Automated AAC Language Activity Monitoring. National Institute for Deafness and Other Communication Disorders SBIR Phase I Grant No. 1 R43 DC04246-01.

Romich, B.A., & Hill, K.J. (2000b). AAC communication rate measurement: Tools and methods for clinical use. In Proceedings of the RESNA '00 Annual Conference. Arlington, VA: RESNA Press. p. 58-60.

Romich, B.A., & Hill, K.J. (2000c). AAC selection rate measurement: Tools andmethods for clinical use. In Proceedings of the RESNA '00 Annual Conference.

Arlington, VA: RESNA Press. p. 61-63.

Contacts:

Katya Hill, MA, CCC-SLP
Assistant Professor
Leader Clinic
Edinboro University of Pennsylvania
Edinboro, PA 16444
Tel: 814-732-2431
Fax: 814-732-2184
E-mail: khill@edinboro.edu

Barry Romich
Prentke Romich Company
1022 Heyl Road
Wooster, OH 44691-9786
Tel: 330-262-1984 ext. 211
Fax: 330-263-4829
E-mail: bromich@aol.com

Summary:

AAC evidence-based practice requires quantitative measurement of communication performance. A set of summary measures is presented together with tools and methods for obtaining them.


Go to previous article 
Go to next article 
Return to 2001 Table of Contents 
Return to Table of Proceedings


Reprinted with author(s) permission. Author(s) retain copyright.