2001 Conference Proceedings

Go to previous article 
Go to next article 
Return to 2001 Table of Contents

Good Vibrations: Using a Tactile Mouse to Convey Page Layout Information to Visually Impaired Computer Users

Dave Offen
Benetech (formerly Arkenstone)
Beth Thomlinson
Freedom Scientific

How can a computer program easily communicate the layout of a printed page to a blind person? The developers of OPENBook have been trying for some time to come up with a viable solution to this problem, and this goal has finally been achieved.

OPENBook is a software package that gives blind and partially sighted people access to printed text. There are now over 25,000 OPENBook users worldwide. OPENBook was originally designed and developed ten years ago by Arkenstone, now known as Benetech. Last June, the rights to OPENBook were purchased by Freedom Scientific, and Freedom Scientific continues to develop OPENBook with assistance from Benetech's engineers. OPENBook's main function has been to decipher the text on the printed page and to deliver the contents in speech, large print, or Braille to blind and low vision readers. However, until now, the visually impaired reader has had difficulty obtaining a useful overview of the page's structure. Their only option has been to read all the words on the page from beginning to end.

Think for a moment about how a person with good vision might approach reading a page from a magazine. With one glance, he can immediately see how many pictures there are, where they are located, if they have captions, how many columns of print are present, and quickly read the titles and section headings. Only after this initial overview will the sighted reader decide if he wants to read more of the page, or skip this page and go on to the next.

The development of Page Layout Description was stymied within Arkenstone until the March 2000 "Technology and Persons with Disabilities" Conference, where a new, inexpensive tactile mouse was on exhibit. The Logitech WingMan(R) Force Feedback Mouse, developed by Immersion, has been on the market less than one year. Its primary customers are video gamers. But this mouse has capabilities that make it possible for a blind person to use a mouse with a computer. It can be programmed to move under its own power, or to simulate the feel of moving across differing textures. Using this mouse, you can actually "touch" different elements on the screen, or the mouse can show you the location of an item of interest by moving your hand there!

Once Arkenstone realized the new mouse's potential, the plans for incorporating Page Layout Description into OPENBook began to crystallize, and the design of this unique and powerful new feature started to take shape. The final design works best with the addition of the Force Feedback Mouse, but it can also be used with an ordinary mouse, or with no mouse at all. This presentation will give the audience a look behind the scenes at the decisions involved in developing the Page Layout capabilities of OPENBook 5.0, and will demonstrate how this new capability of OPENBook can be used by blind and low vision individuals.

OPENBook can collect page layout information and report it to you when you ask for it. We call that Page Layout Description. In addition, you can navigate from element to element. We call that Page Layout Navigation.

What elements can OPENBook identify? We have programmed it to identify the following as unique elements:

If you have the Logitech WingMan Force Feedback Mouse connected, you can get tactile feedback about each page element. All the page layout functions boil down to two needs: the need to have the page described, and the need to navigate based on the description.

There are several important ways the user can get information about the layout of a given page.

In Guided Layout Mode, OPENBook moves the cursor (pointer) to each element while giving you positional information about each element. In Explore Layout Mode, the user moves the cursor to each element (using either a mouse or keyboard commands) and OPENBook then announces it. You get the same information in Explore Mode and Guided Mode, but in Guided Mode, the program guides your movement, while in Explore Mode, you control your own movement.

How does the tactile mouse fit in with this scheme? The Logitech WingMan Force Feedback Mouse, also called a "tactile mouse," was developed by engineers at Immersion, who worked with the OPENBook engineers to integrate the tactile mouse with OPENBook. When you use the WingMan mouse with OPENBook's Page Layout functions, you will get unique tactile feedback for each element type. For example, as you move through a text block you will feel a washboard effect, as though the tactile mouse were bumping over the lines of text. As you move from one column to another, you will feel resistance at the edge of the column. Graphics have a texture like a window screen.

It is important to note that you can use and benefit from Page Layout functions whether you have a tactile mouse, a regular mouse, or no mouse at all. The same goes for visual feedback with a monitor. The primary feedback mode is audio: you will get verbal feedback whether you have a monitor or not. The monitor and mouse don't give more information about the page; they simply give feedback in additional sensory modalities.

Both modes display the entire page in Exact View. Both modes spotlight the entire element as OPENBook moves to the element (the default spotlight color is yellow, but you can set it to a different color).

In both modes, OPENBook uses its popular Exact View, which is a picture of the original page. The entire original page, either in black and white or in full color, is displayed on the screen in Exact View. The computer cursor (pointer) can then move around on the picture of the page, identifying each element as it lands on it. OPENBook will give you simultaneous visual and audio feedback about each element in both modes. If you have a tactile mouse connected, Open Book will give you unique tactile feedback for each element type.

We will demonstrate Page Layout by exploring several scanned pages with the tactile mouse connected. At the end of the session, the audience will be invited to try the tactile mouse themselves.

The goal of Page Layout is the same as the goal of OPENBook: to give visually impaired users the same information about printed material that is available to their sighted co-workers, classmates, and family members. We believe this new technology takes another step toward information equality for visually impaired people.

Go to previous article 
Go to next article 
Return to 2001 Table of Contents 
Return to Table of Proceedings

Reprinted with author(s) permission. Author(s) retain copyright.