2004 Conference Proceedings

Go to previous article 
Go to next article 
Return to 2004 Table of Contents 


Kris Schindler
Michael Buckley
Helene Kershner
University at Buffalo
201 Bell Hall
Buffalo, NY 14260
Email: kds@cse.buffalo.edu 
Email: mikeb@cse.buffalo.edu 
Email: kershner@cse.buffalo.edu 

Students in the computer science and computer engineering programs at the University at Buffalo are developing solutions to socially relevant projects that assist disabled people in the local community. This is being facilitated through a two-course sequence for undergraduate seniors that functions as a capstone design experience. Students further interested in becoming involved in such projects are encouraged to become part of the UB Technology Group. This group extends the projects fielded in these courses and expects to provide reasonably priced, innovative solutions by applying technology and research to socially relevant projects in the community. This paper outlines two of our current projects. The first is the UB Talker, which is an augmentative communications device for the speech-impaired. The second is DISCO,which is a programmable light and sound station for therapists and teachers. This device enhances choice-making and reinforces cause-and-effect related physical, speech, and occupational therapy sessions with physically and ! developmentally impaired children.


In Spring of 2002 the University at Buffalo?s Computer Science and Engineering Department learned of a 43-year old speech-impaired man (David) at a local long-term care facility (Oakwood at Elderwood) who communicated by pointing to letters on a sheet of paper. More advanced communication aids had been adopted but reliability problems caused these devices to be abandoned for the slow, but reliable paper chart. The authors applied for an Educational Technology grant at the University at Buffalo to purchase the hardware and software required to provide computer science and computer engineering students with an opportunity to develop a cost-effective solution for David. There was no lack of interested students and their enthusiasm far surpassed what the authors had ever envisioned. This prompted us to use this as the design project in the department's software engineering course during the Fall 2002 semester. Previously, traditional projects such as the Mars Lander and the ATM were utilized. By using socially relevant projects students combined software engineering with societal benefit. For many of the students, this was their first experience interacting with a disabled person. This experience proved to be a tremendously positive one for everyone involved. The project transformed what would normally be an academic design project to one with a human factor. Suddenly, the motivating factor in the course was no longer the students' grades. They were determined to help David! In one semester, each of the 13 teams of students had developed working prototypes. The effort was so successful that the project was expanded. A working relationship was created with the Center for Handicapped Children's Learning Center in Williamsville, NY. During the Spring 2003 semester, the software engineering class worked on a child version of the UB Talker.

This experience has inspired the authors to continue this work by development of a new capstone design course for the computer engineering program. This consists of a required two-course sequence which focuses on developing and implementing solutions for socially relevant projects in the local community.

In addition to the experience that students gain in these courses, they can further their involvement by joining the UB Technology Group. This research group is dedicated to expanding upon the work developed in the capstone design courses. Part of this group's challenge, is researching ways to solve some of the novel problems that arise with the projects. The group currently has small teams of students each working on an individual problem. For example, with regards to the UB Talker, the speech group is responsible for ensuring that the UB Talker's voice is as humanlike as possible. They are currently looking at implementing emotion, so the user can sound happy, angry, sad, etc. The database group is attempting to categorize spoken phrases, so the preconstructed phrase list can dynamically change as the Talker is in the field. When a user enters a new phrase, and then speaks that phrase multiple times over a relatively short timeframe, that phrase should automaticall! y be added to the list of phrases. The difficult part is correctly categorizing the phrase without human intervention. The interface group is developing novel ways of easily and efficiently interacting with the Talker. For example, the soft-keyboard design that is used to enter a new phrase is was designed to minimize movement. Another feature is the integration of autoscan, which allows a user with very limited motor skills to select an item by scanning items, and selecting the highlighted item by hitting a switch.


UB Talker

The UB Talker is an augmentative communications device to give a voice to those who are speech-impaired. The Talker is designed to run on any PC, laptop, or tablet PC. It utilizes context awareness to help predict what the user wishes to say. This is accomplished by maintaining a database which includes preconstructed phrases, as well as a record of every phrase that has been spoken. When the user speaks, statistics are maintained to indicate what phrases and words have been spoken. Each word and phrase is tagged with a frequency which is used to construct phrase and word completion lists, presenting likely choices to the user based on frequency and time of day. For example, if the user has said, "I want to watch TV. eight times in the past two evenings, this will be presented as a likely phrase at dinner time this evening. This allows a single click to select the phrase, instead of traversing through multiple menus or typing in the phrase. As part of our research we a! re working on automatically categorizing frequently spoken phrases so they become part of our common phrase lists.

The interface interface consists of menu items whereby the user can drill down to preconstructed phrases. Choices on the interface are depicted as either words or images, depending upon the develop level of the user. While these menus provide an efficient means of quickly speaking a common phrase, the user is not restricted to these phrases. A soft-keyboard has been designed which allows the user to type in a phrase with minimal movement when selecting the characters to construct the sentence. For users with severely limited motor skills, an autoscan feature has been implemented. This constantly scans the menu items, highlighting them serially. When the desired item is highlighted, the user can select that item by hitting an augmentative switch (such as a head switch).

The voice of the UB Talker is constructed using a speech synthesizer. Research is currently being conducted on adding emotion to the voice, allowing the user to sound happy, sad, angry, etc.


The DISCO (Disabled Interactive Sensory Coordination Opportunities) system utilizes light, music, and sound to help therapists and teachers create a choice-making, positive feedback, or a calming environment for students who react positively to enhanced sensory experiences. The users are severely handicapped students. As they use the systems, statistics are kept on successes and failures during use, so that teachers and therapists can alter the experience, and use it as part of a child's Individual Education Plan (IEP). The system was dubbed DISCO by our students since the system helping handicapped children with the use of stage lighting, fog machines, fiber optics, water, sound, music, and video.


The CSE department at the University at Buffalo has incorporated the development of solutions to socially relevant projects into their curriculum. This provides students an opportunity to apply research and technology to develop cost-effective, customized solutions to improve the quality of life for those with disabilities. The experience has proved to be a very positive one for everyone involved. The recipients of the technology have an improved quality of life, and the students learn things that cannot be taught in a classroom or conveyed in a textbook. The local community has responded positively, publishing our work in the Buffalo News (local newspaper), Across the Woods (publication of Elderwood), and the Reporter (university publication). The enthusiasm generated by this endeavor has been a motivating factor in expanding this effort at the university and encouraging other colleges and universities to develop similar programs.

Future Work

The research group we formed, called the UB Technology Group is currently developing a lab to further development on the UB Talker and DISCO. We are also actively searching for other socially relevant projects in the community. The objective of this endeavor is to use technology to help as many people as possible while providing students with an opportunity to learn how they can use their skills to apply technology to help those with disabilities. Interested parties are invited to contact us at info@ubtech.cse.buffalo.edu

Go to previous article 
Go to next article 
Return to 2004 Table of Contents 
Return to Table of Proceedings

Reprinted with author(s) permission. Author(s) retain copyright.