2004 Conference Proceedings

Go to previous article 
Go to next article 
Return to 2004 Table of Contents 


A QUANTITATIVE ASSESSMENT OF WEBSITE ACCESSIBILITY FOR VOICE BROWSER USERS

Presenter(s)
Toyohiko Hayashi, Ph.D.
Shingo Aoyagi, B.E.
Yasuo Nakamura, Ph.D.
N. Tondokoro, M.S. Department of Biocybernetics, Faculty of Engineering,
Niigata University
2-8050 Ikarashi, Niigata 950-2181, Japan

1. Introduction
Several guidelines have been proposed in order to make Website more accessible for people with disabilities [1]. To achieve this goal, we have to establish a general method capable of evaluating the accessibility of Websites. Up to now, its qualitative assessment has been discussed extensively, but its quantitative assessment has yet to be established [2,3], primarily because a tremendous amount of factors associated with one another are involved. Then, the authors have been investigated a quantitative assessment of the Website accessibility from a perspective of applied ergonomics for visually impaired persons using a voice browser [4].

The accessibility of Website for such users can be assessed from three independent perspectives as follows: 1) hierarchy of Web-pages; 2) quality and quantity of choice lists; and 3) operationality. In an attempt to elucidate the effects of factors 1) and 2) on the accessibility, we carried out experiments in which subjects were asked to choose a required letter, an alphabetical letter or an integral number, from a Website consisting of several Web-pages each including choice lists, solely by means of a voice browser [3]. Consequently, these factors were verified to affect the accessibility significantly, necessitating us to consider them for developing more concrete guidelines. In previous experiments, however, a regular sequence of choice lists was employed. Then, there is a possibility that such regularity in choice lists would bias empirical results considerably. In order to extract the effects of factors 1) and 2) alone on the Website accessibility, additional experiments were carried out, employing a random sequence of choice lists, and subsequently obtained results were compared with previous data.

2. Materials and Methods
Twenty-six different sequential alphabet-letters (from "a" to "z") and integral numbers (from "1" to "26") were used as data to be selected in a Website. By restricting the number of choice lists to 2, 3, 4, 5 or 26 in each Web-page, we prepared 5 different hierarchical structures for the same Website. For each hierarchical structure, we prearranged two difference sequence patterns of lists, such as regular sequence and random one, where the latter sequence was generated by using a random function in JavaScript. Subjects were asked to complete the following task: "to search for a certain Web-page, in which a required letter is presented, by tracing a sequence of the links from the front page to the destination." For this searching task, a voice browser (Home Page Reader 3.01, IBM Co.) was employed, without using any visualization tools, such as monitor displays. Subjects were allowed to use only "enter" and "return" keys on the keyboard for search and selection. Subjects were 10 males (22.5 years old on average) without any auditory and/or kinetic dysfunctions. Our experimental protocol was as follows:

  1. To provide a subject with a letter to be searched for;
  2. Immediately after Step 1, the subject started to search for the destination Web-page including the required letter;
  3. If the subject arrived at the destination, quit the operation;
  4. The above task from 1) to 3) was repeated 5 times for the same subject by randomly changing a required letter.

The above experiment for each subject was carried out, using 5 different Websites with the afore-mentioned hierarchical structures, each having 2 different sequences of choice lists, regular and random ones. We measured a period of time, denoted as the "access time", between the moment when the start key was pressed and that when the required page was obtained. Then, the "time of operation including thoughts" was computed by subtracting the amount of time spent for reading from the access time. Finally, the "average time of operation including thoughts" was obtained by normalizing the time of operation including thoughts by the number of selections.

3. Results
The statistical difference between two different hierarchical structures of Web-pages was assessed in terms of the average time of operation including thoughts. Regardless of the list sequence, regular or random, the difference between a hierarchical structure with 5 or 26 choice lists included in each Web-page and that with any other number of choice lists was statistically significant. We also assessed the relation between the number of choice lists and the average time of operation including thoughts. Regardless of the number of choice lists, the average time of operation including thoughts regarding integral numbers was shorter than that regarding alphabetical letters. This result was universal, regardless of the list sequence, except the special case with all 26 letters included in only one Web-page.

4. Discussions
When using a voice browser for inspecting a Website, the entire structure of the Website and the understandability of choice lists are believed to strongly affect the accessibility of the Website. Previous guidelines for these factors, however, still remain in a level of practical advices. Universal guiding principles for the factors have yet to be established, probably because the accessibility depends on a tremendous amount of interrelated factors. Then, this paper proposed an experimental protocol dependent solely on the hierarchy of Web-pages and the quality of choice lists, as well as a quantitative evaluation of the accessibility.

In order to establish a required experimental protocol, we have to develop a data set satisfying the following prerequisites: 1) homogeny; 2) nearly the same reading time for all data components; 3) existence of an inherent sequence usable as a clue for search; and 4) without any distinct classification and/or segmentation. If such a data set is obtained, we can easily prepare several empirical Websites with different hierarchical structures. In addition, we need two such data sets with different qualities. As such data sets, we utilized a set of integral numbers and a set of alphabetical letters. The former set, however, do not necessarily satisfy prerequisite 4), due to the use of the decimal system. But its effects on empirical results would probably be negligible, if the number of data is relatively small.

Next, we would like to discuss the evaluation of the accessibility. The access time can be considered a general and quantitative assessment for the accessibility. But it consists of several components as follows: 1) reading time; 2) page-change time; 3) the number of selections; and 4) time for operation including thoughts. Among these components, the last one exclusively reflects the accessibility. Then, we decided to employ the average time of operation including thoughts, which can be obtained by eliminating components 1) through 3) from the entire access time.

The average time of operation including thoughts tended to increase in proportion to the number of choice lists, regardless of both their quality and sequence. Particularly, such increase was statistically significant, when the number of choice lists was larger than or equal to 5. This suggests that the selection becomes complicated as the number of choice lists exceeds 4 on average, under the use of a voice browser, probably due to the difficulty in memory. As mentioned in the previous section, the average time of operation including thoughts regarding integral numbers was usually shorter than that regarding alphabetical letters. This phenomenon would probably originate from the difference in the understandability of the relative position of any link within choice lists.

5. Summary
In order to clarify the effects of the hierarchical structure of Web-pages as well as the quality of choice lists on the accessibility when using a voice browser, we carried out several experiments using Websites with different hierarchical structures and data sets, and quantitatively assessed the accessibility employing the average time of operation including thoughts. Results were summarized as follows:

  1. The accessibility of a Website depends on its hierarchical structure, particularly when the number of choice lists exceeds 4.
  2. The quality of choice lists, particularly the understandability in their relative position, also affects the accessibility.

References

[1] URL: http://www.w3.org/TR/WCAG10/

[2] M. Okada, T. Yamaoka and T. Matsunobe: Structured Website contents construct method and its evaluations, Proc. Conference of Emotion and Human Sensibility, pp.125-130, 2001.

[3] T. Matsunobe and H. Sato: Study on menu usability and structural analysis, Universal Access in HCI, pp.41-45, 2001.

[4] S. Aoyagi, T. Hayashi, Y. Nakamura, and N. Tondokoro: Accessibility evaluation of Website taking into account the use of persons with visual impairments -Effects of the hierarchy and selected-information quality of Website on the access time-, Meeting of Speech and Welfare Information Technology, ISICE, SP2002-116 WIT2002-56, pp.9-16, 2002-10.


Go to previous article 
Go to next article 
Return to 2004 Table of Contents 
Return to Table of Proceedings


Reprinted with author(s) permission. Author(s) retain copyright.