2001 Conference Proceedings

Go to previous article 
Go to next article 
Return to 2001 Table of Contents


The Science Touch System: An Integrated Approach to the Study of Biochemistry and Related Disciplines for Students with Visual Impairments

Dennis Fantin, Ph.D.
Marc Sutton
Scott Luebking

In 1998, the National Science Foundation (NSF) established a three-year research and development project, entitled Design of Physical Models in Biology and Chemistry for Blind University Students, to develop tactile representations of simple molecules and complex macromolecules for use as instructional materials for blind and visually impaired students of biochemistry and related subjects. The principle investigator for the project is Dr. Dennis Fantin, a blind biophysicist at San Francisco State University (SFSU), and visiting research scientist at the Lawrence Berkeley National Laboratory (LBNL).

During his fourteen-year career as a teaching scientist, Dr. Fantin has had many unique opportunities to explore methods for overcoming information barriers faced by blind and visually impaired scientists. This project has been established by the NSF to test a promising method for presenting information, typically contained only in print illustrations, in an information rich tactile format designed to meet the specific needs of blind and visually-impaired students.

The project's two major objectives are:

1) To produce a series of tactile representations of key structures, processes, and relations in biochemistry to be used as instructional aids for blind and visually-impaired students, and:

To develop an accompanying series of computer-based descriptive audio tutorials.

To accomplish these objectives, Dr. Fantin, in conjunction with faculty associates and students at SFSU and LBNL, has developed the Science Touch System, consisting of three-dimensional tactile models, recorded speech tutorials that describe the models and summarize their science, and a sound-file navigation system to facilitate exploration of the tutorials.

The Need for the Science Touch System

In the sciences, much course material is contained in diagrams, illustrations, and charts. Though it is a straightforward process to translate text into Braille, or to record it on tape for students who are blind or visually impaired, information presented in non-text formats is often omitted or inadequately translated. As a result, blind and visually impaired students are hampered in their ability to learn and retain critical information. Such knowledge barriers are particularly profound and numerous in the life sciences and chemistry. To overcome these barriers, blind and visually impaired students are forced to improvise their own methods for gaining access to non-textual information. Typically, these methods involve having readers or friends construct makeshift three-dimensional models out of common materials; a process that takes much time and effort, and comes with the risk that the information will be incorrectly conveyed.

A second major factor that restricts the blind student's efficient access to printed materials relates to the fact that textbooks in the sciences are, by and large, only available in the form of audio tape recordings. Although useful for cereal listening, magnetic tape does not provide any random-access navigation features. Students with visual impairments who must rely on these recordings are thus denied quick and efficient access to the material contained in the textbooks.

Because of the inordinate time spent overcoming visual media barriers such as those described above, blind and visually-impaired students too frequently drop out of the biological sciences or chemistry altogether to explore other fields where the information barriers are less extreme.

How the Science Touch System Works

The Science Touch System has been developed as an integrated approach to the study of biochemistry, molecular biology, and allied disciplines for blind and visually impaired students and scientists. Because the system is intended for the use of people with visual impairments, it is designed to take advantage of the natural perceptual abilities of those who primarily experience the physical world through tactual or, more broadly, haptic inputs, and through sound. The topics selected for inclusion in the system are of universal importance, and include nucleic acid and protein structure; metabolic processes such as the Krebs cycle and glycolysis; and cell cycle and genetic processes such as mitosis, meiosis, and genetic recombination.

These topics are presented through a combination of three-dimensional models and twenty minute spoken word tutorials that provide a detailed guided tour of each model series, and a description of the science underlying the models. The audio tutorials are designed to be played over the PC's sound card, and are controlled either by one hand with all of the navigation commands present on the PC's numeric keypad, or in a hands-free mode with voice commands using a speech recognition program. This built-in flexibility allows the student to either keep one hand anchored on a model while the other hand is left free to control or rapidly navigate through the descriptive narration or, in the hands-free mode, to keep both hands in contact with the physical model while controlling the descriptive narration by voice.

The Models:

In the Science Touch System, molecules are presented in the form of three-dimensional diagrams in which raised shapes or polygons designate the atoms, and flat or curved line segments designate the bond lines. Carbon is represented as a square, Oxygen as a circle, Nitrogen as a triangle, Phosphorus as a pentagon, and Hydrogen as a small hemisphere. Except for Hydrogen, the atoms are represented at the same height above the background, or matrix, in order to simplify model design. The design does, however, convey three dimensional molecular structure information through a haptic projection system developed as an adaptation from the isometric projection method in print illustrations.

In the Science Touch System, flat bond lines are used to connect atoms represented in the plane of the diagram or in a parallel plane, while curved lines are used to connect atoms represented at different heights. The hydrogen atom is treated as a special case because of its monovalent bonding characteristic. Except when hydrogen-bonded, it always exists at the periphery of a molecule and therefore the height of the hemisphere denoting Hydrogen can be raised or lowered depending on the spatial representation intended. Adjusting the height of the hydrogen atom to highlight the direction of the slope of the connecting bond line helps to create and intuitive spatial awareness of the three dimensional molecular configuration.

The models are fabricated out of hard ABS plastic, and resemble articulated puzzle pieces; flat on the bottom, with the chemical structure represented on the surface. The outer edge of each model is cut to conform exactly to the shape of the chemical structure depicted on the surface. Because of this unique design feature, the models can function as tactile illustrations while still retaining properties of real three-dimensional objects. As a further consequence, the models are amenable to a variety of tactual inspection techniques.

When viewed as illustrations, the models can be examined using the tips of the fingers to trace the network of atoms and bond lines on the surface of the structure to gather precise information regarding the identity of atoms, their bonding patterns, and spatial interrelationships between model components. For this type of close, fingertip inspection, the models are typically laid flat on the table with the heel of the hand resting comfortably on the tabletop, while the fingers examine the surface relief.

When viewed as objects, the models can be picked up and explored by the fingers and insides of the hands. This object-level approach facilitates rapid scanning of the periphery of the structure, and provides a more generalized impression, or bird's eye view, of the molecule; including the number and location of rings, the location of chemical bridges and side chains, and the placement and identity of functional groups. An example of object-level design in the Science Touch System is the texturing provided on the inside, outside, and backs of all chemical rings. This provides a means of quickly distinguishing between ring and non-ring components of molecules (the latter being smooth).

Several aspects of the Science Touch System have been designed to promote active manual manipulation of models because active physical exploration is understood to improve memory learning; a fact confirmed by common experience and research from the field of cognitive science. The system incorporates as much physical interchange of model components as possible. For example, enzyme-substrate reactions, such as those occurring in glycolysis, are presented by interlocking the molecules through simple lock and key fittings. Other examples of chemical reactions involving physical interchange of model components include kinase reactions, substrate-level phosphorilations and isomerizations.

The positioning of Braille labels also encourages active manual exploration of the models. The labels are located in recessed pockets in the underside of models requiring that the model be turned over to read the label. The orientation of the Braille labels also defines the absolute orientation of the model for descriptive purposes. The labels are recessed slightly to insure that models lay flat on the tabletop during inspection.

The Science Touch system also provides accurate interatomic bond lengths, defined as the center-to-center distance between atoms measured along the matrix at a scale of 15 mm to 1 angstrom. One cm is the minimum bond line length in the Science Touch System, an approximate finger-width distance, which also serves as a comfortable gage of interatomic separation.
> Extensive testing of specific features of the molecular design in the Science Touch System has been conducted on the intermediates of the glycolytic pathway. Blind and sighted students and faculty associates have evaluated several dozen iterations of each model design feature and component to arrive at a final design. This design is being utilized in the development of several additional model sets in the series.

Although the current project is intended to produce prototype models and designs rather than mass-produced components ready for widespread distribution, commercial partners are currently being sought for the eventual production of the Science Touch models. When this process is completed, the models will be available to all interested blind and low vision students and scientists.

The Tutorials:

Each model set in the Science Touch System is accompanied by a guided audio tour on CDROM that describes the models in detail and provides pertinent scientific background information. The audio tours consist of easy-to-navigate speech recordings played over the PC's sound card using the facilities of the Windows Media Player (versions 7.0 and above). The Media Player, which serves as a rudimentary digital tape recorder, is built into the Internet Explorer browser, which is, in turn, controlled by the Windows Operating System. A major accomplishment of the project has been to design a voice navigation system which expands the capabilities of the Media Player to provide sound-file navigation comparable to text-file navigation already provided by blind-access screen readers. The voice navigation program has been extensively tested, and has been shown to function well in Windows 95 and 98 using Internet Explorer versions 4 and above. The program, written by Scot Luebking as a part of the grant, is written in Javascript and XML, which control the presentation of the sound file and associated text after conversion to HTML.

The tutorials themselves are recorded in the ASF-file format, which allows for the inclusion of embedded navigation marks. These navigation marks are, in turn, used by the voice navigation system to divide files by sentence, paragraph, section, and chapter. Other navigation functions include “move to the beginning or end of the file;” “fast forward” or “rewind” by a specified time increment; “move to an associated linked topic;” and “see the file as text,” in order to check word spellings, perform text-based searches, etc.

The recorded tutorials presented in the Science Touch System are designed to be controlled by one hand, with all of the navigation commands located on the number pad to the right of the main PC keyboard. In this way, one hand can remain anchored on the model while the other hand controls the navigation of the audio tour. Alternatively, hands-free navigation is afforded through speech commands provided through the Dragon Dictate speech recognition system. In hands-free mode, the student speaks the navigation commands aloud. For example, the word "sentence" is spoken to move forward one sentence, and "back sentence" is spoken to move back one sentence in the tutorial. The spoken terms are recognized by Dragon Dictate, which, in turn, activates the appropriate navigation command. For hands-free navigation, the student typically wears a head set with a built-in microphone. Because Dragon Dictate is not a perfect speech recognition engine, with recognition accuracy in about the ninety percent range, the spoken commands occasionally need to be repeated before the action is implemented.

The speech navigation program is completed, and will be provided free upon request. At present, a tutorial describing the glycolytic pathway is available for distribution. The accompanying physical models, though available as prototypes, are not yet available for distribution. Additional tutorials describing the structure of DNA and the model of the periodic table of the elements are currently in a pre-production phase of evaluation.


Go to previous article 
Go to next article 
Return to 2001 Table of Contents 
Return to Table of Proceedings


Reprinted with author(s) permission. Author(s) retain copyright.