2006 Conference General Sessions

Go to previous article
Go to next article
Return to 2006 Table of Contents



Shadi Abou-Zahra
World Wide Web Consortium (W3C)
Web Accessibility Initiative (WAI)
2004, Route des Lucioles
Sophia-Antipolis, France
Email: shadi@w3.org

This presentation highlights some of the key features of Web accessibility evaluation tools and how they can help accomplish different tasks.

Web accessibility evaluation reviews include a variety of checks that are often only partially automatable. Moreover, there is a wide range of Web developers including content authors, graphics designer, programmers, and others. This broad spectrum of developers who want to fulfill different evaluation tasks has lead to the production of many types of tools and tool features. Depending on how rigorous the evaluation review is intended to be, the complexity of the Web site, the experience and knowledge skills of the developers, as well as other considerations, some of these tool features may be more or less relevant in the specific context of the evaluation. This paper highlights some of the overall considerations and some of the evaluation tool features to help developers and procurers decide which tools may best support their needs.

Web accessibility evaluation tools are software programs that help evaluate Web sites for accessibility. These tools are sometimes desktop applications that are installed locally on a computer or online services on remote servers. Furthermore, some of these tools are so called "plug-ins" that integrate with authoring tools (such as editors, content management systems, or word processors) or integrate with browsers. Also assistive technologies such as screen readers or alternative browsers such as text and voice browsers can be used to evaluate accessibility even though they are primarily developed for other purposes.

It is important to note that at this point in time, there is no single Web accessibility evaluation tool that is capable of automatically assessing all accessibility aspects of Web sites without interventions from a human. The reason for this is that accessibility requirements can not always be expressed in formal rules but require human judgement to determine according to a specific context. For example, it is quite simple to automatically check if images are associated with alternate descriptions in HTML but only a human can determine if the description actually makes sense for that image in that specific Web page. However, tools can significantly improve the efficiency of evaluation processes by assisting reviewers carry out checks that can be automated and checks that need to be conducted manually.


There are many different motivations for carrying out Web accessibility evaluation reviews. Ideally, accessibility evaluations are carried out throughout the design, development, and maintenance phases of Web sites. Additionally, more rigorous evaluations are often carried out after a Web site is launched, possibly through a separate quality assurance team or external third-parties. There are also different kinds of roles responsible in the development of accessible Web sites. These roles include Web content authors, graphics designers, programmers, and project managers (depending on the organizational structure and development process, developers may have several responsibilities at once). The following are some consideration to help identify the context in which the evaluation tools will be used in:

* Who will the evaluation tools be primarily for?
   As briefly mentioned, content authors, graphics designers, programmers, and project managers will be often looking for different information when evaluating a Web site for accessibility.

* What skills and experience do these users have?
   Developers who are new to Web accessibility may need more feedback about the findings of the tools while this same information may too verbose for more experienced evaluators.

* How complex is the Web site and how big is it?
   Web sites that have a large number of pages or that make strong use of scripting or advanced Web technologies may present special challenges that are not addressed by all tools.

* Is there an existing development environment?
   Some tools can be integrated into authoring tools (such as editors, content management systems, or word processors) or integrate with browsers; or produce EARL or XML reports.


Due to the broad diversity in users and user requirements, evaluation tool developers have implemented different modalities for interacting with evaluation tools. Some tools provide more than one mode of operation to help evaluators carry out different tasks (for example automatic and manual checking). The following are some these modes that are currently available:

* Generating reports
   Summarize the findings in evaluation reports that are sometimes customizable.

* Step-by-step evaluations
   Carry out checks one at a time either automatically or with the help of a human.

* In-page feedback
   Highlight areas or indicate errors directly on the page to help identify problems.

* Page transformations
   Modify the presentation of the pages such as reading it aloud or enlarging the font.

* Monitoring Services
   Run remotely on the Web server and notify administrators about potential problems.


There are many features provided by Web accessibility evaluation tools and it is beyond the scope of this paper to research each single one. However, the following are broad categories of some of the types of features:

* Accessibility: How accessible is the evaluation tool for people with disabilities?

* Checkpoint coverage: Which checkpoints is the evaluation tool able to adequately address?

* Configuration: How well does the evaluation tool adapt to the requirements of the users?

* Integration: How well does the evaluation tool integrate into the Web development environment of the users?

* Policy requirements: Which guidelines and policy requirements does the evaluation tool support?

* Reliability: How reliable are the results delivered by the evaluation tool?

* Repair: How well does the evaluation tool assist developers in repairing inaccessible Web sites?

* Web technology support: How well does the evaluation tool support the relevant Web technologies?


Web accessibility evaluation tools can significantly improve the efficiency of the evaluation process. However, there are many different types of developers and motivations for carrying out Web accessibility evaluations. It is important to analyze and understand the capabilities of the evaluation tools to get the optimal performance. Also, tools have limitations because accessibility checks are not always automatable. It is also important to understand the underlying accessibility requirements in order to be able to use tools to evaluate them rather than relying on the output of the tools which may sometimes be imprecise.


* Selecting Web Accessibility Evaluation Tools

* Evaluation Tools for Web Content Accessibility

* Web Content Accessibility Guidelines (WCAG)

* Evaluation and Report Language (EARL)

* Web Accessibility Initiative (WAI)

* Education and Outreach Working Group (EOWG)

* Evaluation and Repair Tools Working Group (ERT WG)

Go to previous article
Go to next article
Return to 2006 Table of Contents

Reprinted with author(s) permission. Author(s) retain copyright