2000 Conference Proceedings

Go to previous article 
Go to next article 
Return to 2000 Table of Contents


Usability Testing with Screen Reading Technology in a Windows Environment

Kitch Barnicle
Trace Research and Development Center
5901 Research Park Blvd
Madison, WI 53719
barnicle@trace.wisc.edu

This paper discusses one usability testing technique used to identify and examine obstacles that individuals who are blind and who use screen reading technology encountered while attempting to complete a series of tasks using Windows 95 and associated software applications. The usability testing technique, adapted from the mainstream, was able to generate a valuable set of data that helped explain how both novice and experienced computer users interact with Windows software applications. Software and assistive technology developers, as well as clinicians and researchers may be able to use usability testing techniques to get a better understanding of the needs of users with disabilities.

Introduction

In the mainstream computer industry, a number of techniques are currently being used to assess an application's usability. Many usability studies are conducted with potential product users. In practice, usability studies of mainstream products are usually conducted in three types of situations:

  1. by a manufacturer during the course of product development in order to uncover flaws in the product so that they can be corrected before the product is released;
  2. by an information systems department comparing two or more competing products before making a purchase decision;
  3. by researchers developing a product, comparing products, developing a methodology or investigating human computer interaction.

Thus, usability studies can be characterized by the person or company conducting the study, the stage of product development, and how the study results will be used. Perhaps the most common type of usability study, is one carried out during the development of a software application. These pre-market assessments are carried out so that problems can be uncovered and design improvements can be incorporated into the final product. The results of pre-market usability assessments are often internal company documents that are not publicly available. So, while the presence of users with disabilities in usability studies of mainstream products is believed to be very low, it is difficult to confirm since so few companies make that information available. One goal of this study was to demonstrate the feasibility of conducting a usability study with individuals who used assistive technology in conjunction with mainstream software applications. Questions that arose included a) How must the testing techniques be adapted to accommodate the needs of participants; b) Would the study yield useful data; and c) How will I know if the obstacles encountered were due to the mainstream software application, the assistive technology or the unique characteristics of an individual user?

Methods

A test plan was developed and pilot tested with four individuals. After the pilot tests were completed, a final test plan and associated materials were developed. A key aspect of this plan was a two page checklist that was followed for each data collection session.

Participants were asked to carry out computer-based tasks using Microsoft Word 97, Netscape Navigator 4.04 and Henter-Joyce's JAWS 3.2 for Windows. The tasks included in the study were chosen after considering several factors. The task sets included a combination of what were believed to be both simple and complicated tasks. Five sample tasks are listed below.

  1. Bring the file on the floppy disk called "science.doc" into Microsoft Word.
  2. Set the document's font to Times New Roman, 12 point.
  3. Save the file on the floppy disk.
  4. Set Netscape so web page images do not appear on the screen.
  5. From the Nebraska home page, please go to the "Attractions" page.
  6. Each session was video taped and post-session interviews were audio taped.

Results

The usability testing technique, adapted from the mainstream, was able to generate a valuable set of data that helped explain how both novice and experienced computer users interact with Windows software applications. The process of reviewing the videotape recordings was aided by the fact that participants were using a screen reader and speech synthesizer. From an experimenter's standpoint, tracking participant actions was simplified since the screen reading program and speech synthesizer echoed most user commands.

Using this testing technique it was possible to identify fifty-eight obstacles encountered by participants. Both novice and experienced participants had difficulty completing certain tasks efficiently and successfully; although on average, experienced participants were able to complete more tasks successfully using a fewer number of actions.

Conclusion

Software applications with a graphical user interface have been a major source of concern within the community of computer users who are visually impaired since the first graphical user interface was introduced to the mainstream in the early 1980s. Fifteen years later, despite progress in screen reader development, users of graphical user interface applications who are blind still face many obstacles when using these applications.

There is a great need for data on the usability of software applications from the perspective of users with disabilities. Likewise, the growing importance of the World Wide Web and of information technology in general, make it essential that software and web developers understand the needs of users with disabilities. Usability studies that include users with disabilities can provide developers with tremendous insight on the obstacles encountered by users with disabilities and as a result lead to better designed products.

This study represents only a tiny snapshot of this important area of investigation. Additional studies that a) examine the effects of training, b) take place in "real life" settings, and c) assess the usability of specific interface components, among others, are needed. Mainstream information technology companies and the members of the disability community need to work together to develop and refine techniques for including users with disabilities in usability studies.


Go to previous article 
Go to next article 
Return to 2000 Table of Contents 
Return to Table of Proceedings


Reprinted with author(s) permission. Author(s) retain copyright.