2001 Conference Proceedings
Go to previous article
Go to next article
Return to 2001 Table of Contents
The Science Touch System: An Integrated Approach to the Study
of Biochemistry and Related Disciplines for Students with Visual
Impairments
Dennis Fantin, Ph.D.
Marc Sutton
Scott Luebking
In 1998, the National Science Foundation (NSF) established a
three-year research and development project, entitled Design of
Physical Models in Biology and Chemistry for Blind University
Students, to develop tactile representations of simple molecules
and complex macromolecules for use as instructional materials for
blind and visually impaired students of biochemistry and related
subjects. The principle investigator for the project is Dr.
Dennis Fantin, a blind biophysicist at San Francisco State
University (SFSU), and visiting research scientist at the
Lawrence Berkeley National Laboratory (LBNL).
During his fourteen-year career as a teaching scientist, Dr.
Fantin has had many unique opportunities to explore methods for
overcoming information barriers faced by blind and visually
impaired scientists. This project has been established by the NSF
to test a promising method for presenting information, typically
contained only in print illustrations, in an information rich
tactile format designed to meet the specific needs of blind and
visually-impaired students.
The project's two major objectives are:
1) To produce a series of tactile representations of key
structures, processes, and relations in biochemistry to be used
as instructional aids for blind and visually-impaired students,
and:
To develop an accompanying series of computer-based descriptive
audio tutorials.
To accomplish these objectives, Dr. Fantin, in conjunction with
faculty associates and students at SFSU and LBNL, has developed
the Science Touch System, consisting of three-dimensional tactile
models, recorded speech tutorials that describe the models and
summarize their science, and a sound-file navigation system to
facilitate exploration of the tutorials.
The Need for the Science Touch System
In the sciences, much course material is contained in diagrams,
illustrations, and charts. Though it is a straightforward process
to translate text into Braille, or to record it on tape for
students who are blind or visually impaired, information
presented in non-text formats is often omitted or inadequately
translated. As a result, blind and visually impaired students are
hampered in their ability to learn and retain critical
information. Such knowledge barriers are particularly profound
and numerous in the life sciences and chemistry. To overcome
these barriers, blind and visually impaired students are forced
to improvise their own methods for gaining access to non-textual
information. Typically, these methods involve having readers or
friends construct makeshift three-dimensional models out of
common materials; a process that takes much time and effort, and
comes with the risk that the information will be incorrectly
conveyed.
A second major factor that restricts the blind student's
efficient access to printed materials relates to the fact that
textbooks in the sciences are, by and large, only available in
the form of audio tape recordings. Although useful for cereal
listening, magnetic tape does not provide any random-access
navigation features. Students with visual impairments who must
rely on these recordings are thus denied quick and efficient
access to the material contained in the textbooks.
Because of the inordinate time spent overcoming visual media
barriers such as those described above, blind and
visually-impaired students too frequently drop out of the
biological sciences or chemistry altogether to explore other
fields where the information barriers are less extreme.
How the Science Touch System Works
The Science Touch System has been developed as an integrated
approach to the study of biochemistry, molecular biology, and
allied disciplines for blind and visually impaired students and
scientists. Because the system is intended for the use of people
with visual impairments, it is designed to take advantage of the
natural perceptual abilities of those who primarily experience
the physical world through tactual or, more broadly, haptic
inputs, and through sound. The topics selected for inclusion in
the system are of universal importance, and include nucleic acid
and protein structure; metabolic processes such as the Krebs
cycle and glycolysis; and cell cycle and genetic processes such
as mitosis, meiosis, and genetic recombination.
These topics are presented through a combination of
three-dimensional models and twenty minute spoken word tutorials
that provide a detailed guided tour of each model series, and a
description of the science underlying the models. The audio
tutorials are designed to be played over the PC's sound card, and
are controlled either by one hand with all of the navigation
commands present on the PC's numeric keypad, or in a hands-free
mode with voice commands using a speech recognition program. This
built-in flexibility allows the student to either keep one hand
anchored on a model while the other hand is left free to control
or rapidly navigate through the descriptive narration or, in the
hands-free mode, to keep both hands in contact with the physical
model while controlling the descriptive narration by voice.
The Models:
In the Science Touch System, molecules are presented in the form
of three-dimensional diagrams in which raised shapes or polygons
designate the atoms, and flat or curved line segments designate
the bond lines. Carbon is represented as a square, Oxygen as a
circle, Nitrogen as a triangle, Phosphorus as a pentagon, and
Hydrogen as a small hemisphere. Except for Hydrogen, the atoms
are represented at the same height above the background, or
matrix, in order to simplify model design. The design does,
however, convey three dimensional molecular structure information
through a haptic projection system developed as an adaptation
from the isometric projection method in print illustrations.
In the Science Touch System, flat bond lines are used to connect
atoms represented in the plane of the diagram or in a parallel
plane, while curved lines are used to connect atoms represented
at different heights. The hydrogen atom is treated as a special
case because of its monovalent bonding characteristic. Except
when hydrogen-bonded, it always exists at the periphery of a
molecule and therefore the height of the hemisphere denoting
Hydrogen can be raised or lowered depending on the spatial
representation intended. Adjusting the height of the hydrogen
atom to highlight the direction of the slope of the connecting
bond line helps to create and intuitive spatial awareness of the
three dimensional molecular configuration.
The models are fabricated out of hard ABS plastic, and resemble
articulated puzzle pieces; flat on the bottom, with the chemical
structure represented on the surface. The outer edge of each
model is cut to conform exactly to the shape of the chemical
structure depicted on the surface. Because of this unique design
feature, the models can function as tactile illustrations while
still retaining properties of real three-dimensional objects. As
a further consequence, the models are amenable to a variety of
tactual inspection techniques.
When viewed as illustrations, the models can be examined using
the tips of the fingers to trace the network of atoms and bond
lines on the surface of the structure to gather precise
information regarding the identity of atoms, their bonding
patterns, and spatial interrelationships between model
components. For this type of close, fingertip inspection, the
models are typically laid flat on the table with the heel of the
hand resting comfortably on the tabletop, while the fingers
examine the surface relief.
When viewed as objects, the models can be picked up and explored
by the fingers and insides of the hands. This object-level
approach facilitates rapid scanning of the periphery of the
structure, and provides a more generalized impression, or bird's
eye view, of the molecule; including the number and location of
rings, the location of chemical bridges and side chains, and the
placement and identity of functional groups. An example of
object-level design in the Science Touch System is the texturing
provided on the inside, outside, and backs of all chemical rings.
This provides a means of quickly distinguishing between ring and
non-ring components of molecules (the latter being smooth).
Several aspects of the Science Touch System have been designed
to promote active manual manipulation of models because active
physical exploration is understood to improve memory learning; a
fact confirmed by common experience and research from the field
of cognitive science. The system incorporates as much physical
interchange of model components as possible. For example,
enzyme-substrate reactions, such as those occurring in
glycolysis, are presented by interlocking the molecules through
simple lock and key fittings. Other examples of chemical
reactions involving physical interchange of model components
include kinase reactions, substrate-level phosphorilations and
isomerizations.
The positioning of Braille labels also encourages active manual
exploration of the models. The labels are located in recessed
pockets in the underside of models requiring that the model be
turned over to read the label. The orientation of the Braille
labels also defines the absolute orientation of the model for
descriptive purposes. The labels are recessed slightly to insure
that models lay flat on the tabletop during inspection.
The Science Touch system also provides accurate interatomic bond
lengths, defined as the center-to-center distance between atoms
measured along the matrix at a scale of 15 mm to 1 angstrom. One
cm is the minimum bond line length in the Science Touch System,
an approximate finger-width distance, which also serves as a
comfortable gage of interatomic separation.
> Extensive testing of specific features of the molecular
design in the Science Touch System has been conducted on the
intermediates of the glycolytic pathway. Blind and sighted
students and faculty associates have evaluated several dozen
iterations of each model design feature and component to arrive
at a final design. This design is being utilized in the
development of several additional model sets in the series.
Although the current project is intended to produce prototype
models and designs rather than mass-produced components ready for
widespread distribution, commercial partners are currently being
sought for the eventual production of the Science Touch models.
When this process is completed, the models will be available to
all interested blind and low vision students and scientists.
The Tutorials:
Each model set in the Science Touch System is accompanied by a
guided audio tour on CDROM that describes the models in detail
and provides pertinent scientific background information. The
audio tours consist of easy-to-navigate speech recordings played
over the PC's sound card using the facilities of the Windows
Media Player (versions 7.0 and above). The Media Player, which
serves as a rudimentary digital tape recorder, is built into the
Internet Explorer browser, which is, in turn, controlled by the
Windows Operating System. A major accomplishment of the project
has been to design a voice navigation system which expands the
capabilities of the Media Player to provide sound-file navigation
comparable to text-file navigation already provided by
blind-access screen readers. The voice navigation program has
been extensively tested, and has been shown to function well in
Windows 95 and 98 using Internet Explorer versions 4 and above.
The program, written by Scot Luebking as a part of the grant, is
written in Javascript and XML, which control the presentation of
the sound file and associated text after conversion to HTML.
The tutorials themselves are recorded in the ASF-file format,
which allows for the inclusion of embedded navigation marks.
These navigation marks are, in turn, used by the voice navigation
system to divide files by sentence, paragraph, section, and
chapter. Other navigation functions include “move to the
beginning or end of the file;” “fast forward”
or “rewind” by a specified time increment;
“move to an associated linked topic;” and “see
the file as text,” in order to check word spellings,
perform text-based searches, etc.
The recorded tutorials presented in the Science Touch System are
designed to be controlled by one hand, with all of the navigation
commands located on the number pad to the right of the main PC
keyboard. In this way, one hand can remain anchored on the model
while the other hand controls the navigation of the audio tour.
Alternatively, hands-free navigation is afforded through speech
commands provided through the Dragon Dictate speech recognition
system. In hands-free mode, the student speaks the navigation
commands aloud. For example, the word "sentence" is spoken to
move forward one sentence, and "back sentence" is spoken to move
back one sentence in the tutorial. The spoken terms are
recognized by Dragon Dictate, which, in turn, activates the
appropriate navigation command. For hands-free navigation, the
student typically wears a head set with a built-in microphone.
Because Dragon Dictate is not a perfect speech recognition
engine, with recognition accuracy in about the ninety percent
range, the spoken commands occasionally need to be repeated
before the action is implemented.
The speech navigation program is completed, and will be provided
free upon request. At present, a tutorial describing the
glycolytic pathway is available for distribution. The
accompanying physical models, though available as prototypes, are
not yet available for distribution. Additional tutorials
describing the structure of DNA and the model of the periodic
table of the elements are currently in a pre-production phase of
evaluation.
Go to previous article
Go to next article
Return to 2001 Table of Contents
Return to Table of
Proceedings
Reprinted with author(s) permission. Author(s) retain copyright.