2000 Conference Proceedings

Go to previous article 
Go to next article 
Return to 2000 Table of Contents


Frametalker: Development of a Frame-based Communication System

D. Jeffery Higginbotham, Ph.D.
Department of Communicative Disorders and Sciences
122 Cary Hall, 3435 Main Street, University at Buffalo
Buffalo, NY 14214-3005
Phone: 716-829-2797 ext. 635
Email: cdsjeff@buffalo.edu

Bryan J. Moulton
Enkidu Research, Inc.
24 Howard Avenue
Lockport, NY 14094
Phone: 716-433-0608
Fax: 716-433-6164
Email: moulton@enkidu.net

Gregory W. Lesher, Ph.D.
Enkidu Research, Inc.
Email: lesher@enkidu.net

David P. Wilkins, Ph.D.
Cognitive Anthropology Group
Max Planck Institute for Psycholinguistics
PB-310, 6500 AH Nijmegen
The Netherlands
Email: wilkins@mpi.nl

Jennifer Cornish, B.S.
Department of Linguistics
University at Buffalo
Buffalo, NY 14260
Email: jcornish@acsu.buffalo.edu



Frametalker is an utterance-based communication device utilizing a Communication Frame approach to allow the selection of natural language utterances. A communication frame is an informational organization representing the situational structure off communication events. This organization can be used to organize semantically and functionally related utterance that tend to arise in typical instances of talk.

The impetus for Frametalker comes from an attempt by Higginbotham and Wilkins to provide a strategy for organizing expressive communications utilizing situated activity as a basis for its organization (Higginbotham & Wilkins, 1996). This work is predicated on the view that talk is a type of action, and as action can be represented and is interpreted in a holistic fashion and with respect to the context in which it is used. Recent work by the entire research team has focused on the implementation of the communication frame idea as an utterance-based augmentative communication device that will allow its user to communicate quickly and effectively during various communication events.

Informational Description

A communication frame consists of several parts including component frames, utterance constructions and lexical fields, and a topic domain and frame hierarchy. Each communication frame’s internal schematic structure involves component frames and utterance constructions. Component frames are smaller sets of meaning structures that are dedicated to uniquely identifiable typical subtopics or distinct situational portions within the larger communication frame (e.g., ‘severity’ versus ‘cause’ of pains). Utterance constructions are located within component frames. Each utterance construction in combination with its associated lexical field (i.e., group of semantically related terms) can be used to generate a potentially large number of different utterances. Component frames and utterance constructions are organized within a communication frame to reflect the communication activities of the associated event. Finally, a cluster of individual communication frames that share similar generic topic interests that share a degree of internal organization are organized hierarchically into topic domains. The graphics below illustrate the Frametalker constituents:

Illustration of the Frametalker constituents: A communication frame consists of several parts including component frames, utterance constructions and lexical fields, and a topic domain and frame hierarchy. Each communication frame’s internal schematic structure involves component frames and utterance constructions. Component frames are smaller sets of meaning structures that are dedicated to uniquely identifiable typical subtopics or distinct situational portions within the larger communication frame (e.g., ‘severity’ versus ‘cause’ of pains). Utterance constructions are located within component frames. Each utterance construction in combination with its associated lexical field (i.e., group of semantically related terms) can be used to generate a potentially large number of different utterances. Component frames and utterance constructions are organized within a communication frame to reflect the communication activities of the associated event. Finally, a cluster of individual communication frames that share similar generic topic interests that share a degree of internal organization are organized hierarchically into topic domains.

Illustration of the Frametalker constituents: A communication frame consists of several parts including component frames, utterance constructions and lexical fields, and a topic domain and frame hierarchy. Each communication frame’s internal schematic structure involves component frames and utterance constructions. Component frames are smaller sets of meaning structures that are dedicated to uniquely identifiable typical subtopics or distinct situational portions within the larger communication frame (e.g., ‘severity’ versus ‘cause’ of pains). Utterance constructions are located within component frames. Each utterance construction in combination with its associated lexical field (i.e., group of semantically related terms) can be used to generate a potentially large number of different utterances. Component frames and utterance constructions are organized within a communication frame to reflect the communication activities of the associated event. Finally, a cluster of individual communication frames that share similar generic topic interests that share a degree of internal organization are organized hierarchically into topic domains.  

Communication Frame Development

Communication frames are derived from the empirical analysis of communication situations encountered by physically normal and physically challenged individuals. Repeated observations of specific communication situations (e.g., going to Wendy's or Burger King) are performed and documented by video and audio recordings. These data are then transcribed and analyzed to reveal the event structure and utterance types used to accomplish the event. Once arrived upon, individual utterance constructs are selected and entered into the database. These utterances are entered as abstractions in order to permit shifts in temporal and interpersonal focus. Lexical fields are organized to extend the functionality of particular utterances.

Frametalker Operation

Instantiating a frame involves creating a record containing component frames and messages pertinent to the communication frame. Through a recursive process involving semantic and pragmatic specification, the messages are assembled into component frames, which are in turn, assembled into communication frames. The default features of any particular frame will be matched to the appropriate pragmatic, topic and semantic specificity dictated by the situation.

The means by which a user would make a selection (e.g., touch item, select button, type code) depends on the particular means of access and user interface employed. Frametalker is designed to be independent of any specific language system design and user interface to ensure that the communication frame approach is not artificially constrained by the limitations of a particular graphic or informational representation. Frametalker can be integrated with current augmentative user interfaces, symbol systems and coding techniques.

The graphic below displays the communication frame for ordering and eating at a Burger King (R) restaurant. Utterances are organized according to the event structure at Burger King (in-line > at counter > at table) which is supplemented by other important contextually related talk (e.g., Help from others, size and amounts). The communicator may select from any of the default utterances, or modify the existing utterance by altering the default lexical item (underlined), or shift the temporal (present, past, future) or interpersonal focus (me, you, them) of the utterance by selecting the appropriate modification buttons. The communicator may also access a related communication frame by selecting a frame shift button (bottom of the screen) or expand a component frame by selecting the topic label. Once the utterance is selected it is spoken via a synthetic voice. Frametalker is implemented on the Impact software platform developed by Enkidu Research Inc.

Graphic displaying the communication frame for ordering and eating at a Burger King (R) restaurant. Utterances are organized according to the event structure at Burger King (in-line > at counter > at table) which is supplemented by other important contextually related talk (e.g., Help from others, size and amounts). The communicator may select from any of the default utterances, or modify the existing utterance by altering the default lexical item (underlined), or shift the temporal (present, past, future) or interpersonal focus (me, you, them) of the utterance by selecting the appropriate modification buttons. The communicator may also access a related communication frame by selecting a frame shift button (bottom of the screen) or expand a component frame by selecting the topic label. Once the utterance is selected it is spoken via a synthetic voice. Frametalker is implemented on the Impact software platform developed by Enkidu Research Inc.

Frametalker Assessment

The Frametalker system is currently being assessed in terms of its operability, usability, efficiency and effectiveness as an augmentative communication aid. A disciplined evaluation of the Frametalker system will occur across several levels throughout the project life-cycle. First the operation of the device will be carefully assessed against the specifications established at the beginning of the project. Next, a set of device an user performance assessments will take place in which both quantitative and qualitative data will be collected to provide a comprehensive appraisal used to aid in ongoing system refinement and to establish benchmarks for Frametalker performance. These benchmarks will be used to document the efficiency of the Frametalker system and to compare the performance characteristics of other commercially available augmentative communication devices.

In a preliminary analysis of selection efficiency, Frametalker was compared with a word predictor (CoWriter) and Prentke Romich's Unity (R). To perform the test, utterances were selected randomly from each Frametalker's six ‘windows’ so that roughly 10 utterances were chosen per window. Within these 10 utterances, 4 were kept unaltered. In each window, 6 utterances were altered by changing the temporal or interpersonal focus or the lexical item, resulting in the added selections. The level of machine efficiency was measured in selections (or keystrokes) saved compared to typing out the sentences. Selection savings results are as follows: Frametalker achieved an 87%, CoWriter = 45%; Unity = 40%. A Tukey multiple comparison test of the mean differences in selection savings showed Frametalker to be significantly greater than either of the other two devices (Tukey q=2.952, df=59, p<.01). Although biased towards Frametalker's utterance database, this test clearly demonstrates the device's comparative efficiency in handling situated communications.

Reference

 Higginbotham, D.J. & Wilkins, D.P. (1996). Frametalker: A system and method for utilizing communication frames for augmentative communication. Patent received September 21, 1999. Patent No. 5,956,667.

Acknowledgements

The authors wish to acknowledge support from the National Cancer Institute under grant #R4 CA80715-01 from the National Institutes of Health . The opinions expressed are those of the authors and do not necessarily reflect those of the supporting agency.


Go to previous article 
Go to next article 
Return to 2000 Table of Contents 
Return to Table of Proceedings


Reprinted with author(s) permission. Author(s) retain copyright.