2004 Conference Proceedings

Go to previous article 
Go to next article 
Return to 2004 Table of Contents 


DIGITAL TALKING BOOKS FOR MATH AND SCIENCE

Presenters
David Schleppenbach
Abraham Nemeth
gh, LLC
3000 Kent Ave.
West Lafayette, IN 47906
Website: http://www.ghbraille.com 
Email: engage@ghbraille.com

This paper is a summary of research currently underway to develop a standard for the production of Digital Talking Book versions of Math and Science Books (using MathSpeak), and a software player (the gh PLAYER™) module that can properly render the math and science both aurally and visually. Specifically, several mathematics textbooks, one science textbook, and a standardized test involving math and science have been prepared in this format by gh and tested among print disabled students in a variety of settings.

Several key topics are pursued in this paper, including Information-Based Technologies (IT), Human/Computer Interactions, and Universal Design/Universal Access. This area encompasses hardware and software platforms that empower individuals who have disabilities, enhancing their capabilities in the fields of mathematics, science, and engineering both in the classroom and on the job. Obviously the development of a method to teach math and science independently to students who cannot read the ordinary print materials is a vital and much-needed concept along these lines. In addition, the core principle of the gh PLAYER™ is Universal Design, meaning that all students, regardless of disability, have access to the Math and Science content provided.

This project is significant as the first attempt by an organization to adapt core science and mathematics curricula into the ANSI/NISO Digital Talking Book format for use by all print disabled students. Additionally, this is the first effort to incorporate MathSpeak™ into DTB's and the DAISY/NISO DTB standard.

There is a strong need for better math and science education among print disabled students. Currently, secondary school age students are in large part unable to receive accessible instructional materials in mathematics and science. The natural result of this lack of materials is a general deficit in knowledge and skills and in math and science by students who are print disabled. This lack of accessible instructional materials has unfairly limited students who are print disabled in that they often lack the opportunity to study math and science. Because of a lack of access to science and math instructional materials the print disabled are ultimately unable to seek employment in the growing high tech job sector. The unemployment rate among the working age adults who are print disabled is over 74% (AFB Sept./Oct. JVIB Kirchner & Schneidler, 1997). If print disabled individuals have an equal opportunity to study mathematics and science, many new career opportunities will be opened up to this class of individuals that were previously not viable options.

The ultimate goal of this project is to provide print disabled high school students an opportunity to learn mathematics and science. To further this goal, gh is developing a module for the gh PLAYER™ capable of interpreting MathSpeak™ and rendering both visually and aurally the information encoded therein. This MathSpeak™-compliant software player will be a key commercialization outcome of the project. Although the MathSpeak™ specification itself is intended to be an open specification, there are some aspects of this project which are proprietary, including the gh PLAYER™ itself, which has unique parsing and rendering features so as to make a viable and competitive commercial product.

Additionally, by working with Dr. Nemeth, gh is continuing to refine and codify the MathSpeak™ specification, incorporating the use of both VoiceXML and MathML in the execution of the desired outcome. The eventual result will be a robust specification for spoken math, which will be introduced as a potential addition into both the National File Format standards for K-12 Instructional Materials, and both the ANSI/NISO and DAISY specifications for Digital Talking Books. gh is also committed to is making MathSpeak™ available as an open specification for anyone to build from.

The cornerstone technology of the project is the gh PLAYER™. The gh PLAYER™ integrates six types of accessible media - eBraille, eLarge Print, Digital Talking Books, Foreign Language, Captioning, and Sign Language - allowing a broad range of people access to information sources from textbooks and publications to training manuals.

The gh PLAYER™ allows everyone to have access to print materials. Because the player supports six types of accessible media developed for a variety of needs, this all-in-one reading source proves to be useful for people with disabilities as well as those without, making the technology useful as a mainstream application. Whether a reader needs electronic Braille or a Digital Talking Book, it's a simple matter to select the appropriate interface, open the book from the e-library and begin reading.

The gh PLAYER™ is also very cost-effective as a solution for print disabled students. Because all six media types spawn from the same XML code base, and are produced by the Media Conversion Process™, the school only has to buy one book for everybody. And since it's all on one CD and the reader can just use an existing PC, there's no need to buy any new equipment - let alone six different types of books. Furthermore, the electronic file format eliminates the cost of printing and distribution, thus saving the school even more money.

Finally, the gh PLAYER™ utilizes an electronic media file format which defines portability. When the reader wants to take the book to class or work late at home, they can. There's no clunky hardware or cumbersome stacks of books to carry. It's just a CD and it's just that convenient.

The core technology of the gh PLAYER™ and all of its six media centers around three primary technologies that are put to work behind the scenes to generate its inherent horsepower.

First, gh utilizes the Media Conversion Process™ to create the accessible media. Simply put, this is how gh makes the media accessible. It's a flexible and unique XML-based process that takes a variety of inaccessible print or electronic text formats and produces the desired accessible media - whether it is eBRL, DTBs, eLP, or others. It starts with sending the book to gh and ends with a rich and powerful accessible media on a single CD.

Second, and perhaps most important, the most revolutionary concept of the gh PLAYER™ is the idea of synchronized multimedia where audio, video, text and images play precisely in concert with each other. For instance, the reader can follow the highlighting of each word in a Digital Talking Book while listening to the corresponding audio speech output. The reader can even show an image while voicing a descriptive narrative. A person who is deaf may watch an animated interpretation in sign language while following the text captioning. Synchronized multimedia even provides the power to display English text with Spanish audio. This is a central feature of the ANSI/NISO Digital Talking Book standard and is a key reason why DTB's are such an effective solution for print disabled students.

Finally, gh PLAYER™ utilizes a unique concept in the media file format itself known as semantic markup. When there is a need to convey context-sensitive meaning in a document, semantic markup allows the document author and reader to extract the same meaning from the book or test. All gh PLAYER™ media is designed to utilize semantic markup, which allows words, phrases, and other structures to be marked with meaning as well as format. Now foreign languages can be displayed in Subject/Predicate/Object format, Braille translation can be exact, and even American Sign Language can be displayed with true prosody. Semantic Markup allows readers to easily switch between disparate formats at the touch of a button.

Although the gh PLAYER™ technology and the DTB media type have already been developed by gh, there is an incomplete portion of the ANSI/NISO and DAISY 3.0 standards that limits the utility of these solutions. This incompleteness is that Math and Science information is not currently addressed as part of the DTB solution. gh has developed, in conjunction with Dr. Abraham Nemeth, a technological solution for voicing mathematical and scientific information called MathSpeak™. This work is an extension of the basic MathSpeak™ grammar and lexicon originally developed by Dr. Nemeth for use by a human reader. The basic principle of MathSpeak™ is that print mathematics can be rendered aurally in parallel with both the print and Braille versions of the text. The sample books and test converted for this project demonstrate the ease of use and power of the MathSpeak™ standard.

In sum, gh is working hard to ensure a bright future for students with print disabilities. By developing new technological solutions for the aural rendering of math, gh hopes to provide better access to math and science for students with disabilities. Over the course of this ongoing research project gh has had the opportunity to work with numerous students and to witness positive impact on the lives of disabled students - an impact that will hopefully carry forth for years yet to come.


Go to previous article 
Go to next article 
Return to 2004 Table of Contents 
Return to Table of Proceedings


Reprinted with author(s) permission. Author(s) retain copyright.