Go to previous article
Go to next article
Return to 2005 Table of Contents
Presenter(s)
Elizabeth Bryant
Georgia Tech
Center for Assistive Technology & Environmental Access
490 Tenth Street
Atlanta, Georgia 30318
Phone: (404) 894-0254
Email: bethb@catea.org
Summary
This report documents the results of a usability study conducted on a Web site, the assistivetech.net site[1], designed to meet W3C Web Content Accessibility Guidelines. The study was a two-part process conducted over one year. The purpose of the usability study was to determine the degree to which the design, content, navigation, and search features were accessible and usable by a wide array of users. Users included general adult population, people over the age of 65, adults with visual, hearing, and mobility impairments. The study results provided valuable information regarding human-computer interface design, the application of W3C Web accessibility guidelines, and the need to go beyond accessibility to consider usability.
Description
The usability study was conducted in two phases, the initial phase in September 2003 and the final phase in September 2004. The purpose of the two phases was to make an initial assessment of the Web site's accessibility and usability, make revision based upon the results, and have the second assessment confirm that the changes improved the accessibility and usability. During the initial assessment phase, four participants under each of the seven target audiences: general consumer, people over the age of 65, people with visual impairments, people with hearing impairments, people with mobility impairments, rehabilitation service providers, and AT vendors. Seven scenarios, one for each target audience, were developed to assess the site's usability. Each scenario involved nine tasks; each of the scenarios required participants to perform the same fundamental tasks with slight adjustments in specific situational content. The tasks are listed below.
| Task1 | Examine the assistivetech.net homepage and provide insights |
| Task2 | Spend five minutes exploring the site and comment on what you experience |
| Task3 | Find products using the Vendor Search |
| Task4 | Find a particular type of product using the Keyword Search |
| Task5 | Find a particular type of product using the Product Type Search |
| Task6 | Find a particular type of product using the Function / Activity Search |
| Task7 | Find particular resources |
| Task8 | Use any search tool to find a product of interest |
| Task9 | Locate contact information and the Site Usage Survey. Complete the survey. |
Participants were observed and videotaped while participating in the study. The study facilitator encouraged participants to think out-loud while performing each task. The goal was to capture the participants' thought processes, expectations, observations, challenges and successes while navigating the site and using the site's search tools. As participants spoke, comments were captured, creating a log of that documented key findings. After the initial study was conducted, the findings were compiled and assigned a priority for action. Although some findings were immediate, others were revised over the year between the initial phase and final phase for the study. Revisions were made to address the findings. Included in the revisions were changes to the site's information classifications, terminology, page content, product groupings, internal navigation features and additional usability features. Study findings were categorized into eight primary areas (See Table 1). Each of the finding categories mapped to the Web site's design for achieving accessibility and usability.
Table 1: Finding Categories
Category
Description
Design
Structure and purpose of the site
Navigation
How the user works through the site
Terminology
Word/term usage
Content
Presentation and substance of the content
Point of View
Perspective of the user; what user expects
Positive
Positive comments about features
Observation
General observations
Feedback
Issue with feedback to the user
Bug
System problem or failure
The second study, repeated to determine whether usability and accessibility improvements resulted from the revision, was conducted in September 2004 with five user groups; people from the general adult population and AT vendors were not used in the second study. The same scenarios with nine tasks were used. No user from the first study was recruited to participate in the second study.
Results The two-part study results provided valuable information regarding human-computer interface design, the application of W3C Web accessibility guidelines, and the need design beyond accessibility to consider usability through the user experience.
Reference
[1] www.assistivetech.net
[2] Benbunan-Fich, R. (2001) Using Protocol Analysis to Evaluate the Usability of a Commercial Web Site, Information & Management, 39, 151-163
[3] Dix, A.J. (1998) Human-Computer Interaction, Prentice Hall, London
[4]Nielson, J. (2000) Designing Web Usability, New Riders Publishing, Indiana
[5] Nielson Norman Group (2001) How to Conduct Usability Evaluations for Accessibility
[6] Rubin, J. (1994) Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests, John Wiley & Sons, Inc. New York
[7] W3C (1999) Web Content Accessibility Guidelines, http://www.w3.org/TR/WAI-WEBCONTENT/
Go to previous article
Go to next article
Return to 2005 Table of Contents