1998 Conference Proceedings

Go to previous article. 
Go to next article. 
Return to 1998 Conference Table of Contents


TACTILE IMAGE PROCESSING WITH TACFAX

Thorsten Puck
Harz University Of Applied Studies
Fachbereich Elektrotechnik/Informatik
Friedrichstrasse 57-59
GERMANY
PHONE: +49 3943/659 320
FAX: +49 3943/659 107
EMAIL: mailto:tpuck@fh-harz.de

Introduction

Telephones and fax machines are some of the most used electronic devices - they provide an easy way to reach a communication partner throughout the world.

Blind people have limited access to this sort of document through document reading machines. After a conversion of the documents image to text has taken place a speech synthesiser reads out faxes.

A new tactile printer, developed in the European project PRINT, together with the fax-software TACFAX will allow not only to print faxes in Braille and Moon in a cost-effective way but also to integrate some graphics in the fax document. Nearly all kind of tactile graphics are made for blind people by sighted people [3] [2]. The need to handle faxes and the benefits from being able to communicate through a fax device using graphics challenges also blind people. However printing faxes tactile on paper makes them more comparable to their ink-print counterpart.

With TACFAX, faxes can be stored, carried around and read like any paper document. In addition, unlike a normal fax, we propose to allow mark-up facilities. The user assigns two keywords to each page which will be printed at the top on the page. In order to reduce typing effort, the fax system tries to make a guess based on the phone number to select the keyword to be used. With this method faxes from the same phone number will be assigned the same keyword. This mark-up of the page is additionally used for archiving purposes. Faxes are stored in format HTML, images in GIF.

Recognition of Graphics with OCR

The graphic elements are determined in fax during optical character recognition (OCR). Therefor the document is partitioned in squared sections of text, tables or graphic elements. For determining the sections, first the text lines will be determined by OCR. Therefor most OCR-programs create a histogram of an horizontal projection of black pixels for each scanning line of the page [6]. Text lines are determined through an accumulation of pixels in the histogram. The number of pixels between text lines is ideally zero.

Graphic elements in documents are determined by the same method. Ideally graphics in fax documents consists of a grey scaled pixel matrix, so they can be recognised by analysis of an increased number of pixels in the histogram. Therefore the sections of graphical elements in documents can be created. This can occur mistakes, especially if graphic elements include text.

3.2.2.1 Consequently we have investigated the recognition of graphics by different types of graphics like logo, emblem, graphics with coloured background, hand written graphics, pictures or photographs.

3.2.2.2 Some tests causes the result that logos or emblems of written text affects the process of graphic recognition because their histogram will be similar to the histogram of text elements. On the other hand, graphics with coloured background including text, pictures and photographs are recognised as graphic element because they consists of a pixel matrix.

3.2.2.3 Depending on their motives, some special hand written graphics were recognised. A reason for misrecognition may be that a horizontal painted line is ‘confused’ with the underline of text. Furthermore hand written graphics may not cause a striking value in histogram because of white backgroundcolor. Consequently they are rarely recognised.

3.2.2.4 Diagrams often include text for description and white backgroundcolor. For this reason the OCR may detect several text lines and areas of assuming graphics in diagrams. So diagrams are often split in sections of text and graphic elements.

e. For this reasons, TACFAX offers the possibility to retrieve and reprint faxes by selecting the printing mode. Selecting the sections of text and graphics in a document can then manually be defined again.

Hardware scenarios

The fax system can be used in two scenarios: as stand-alone fax machine and in combination with a desktop computer, based on the operating system MS Window 95. In addition to the software package, the tactile printer will be available as separate device, connected to the parallel port of a PC.

In both scenarios, the fax system is connected to the telephone network and is handled like a standard fax machine including the features of TACFAX.

The connection to a personal computer via Ethernet or null-modem-cable, allows the user to work on recognized text or graphics with different software programs even on a remote PC.

The scenario with TACFAX on a PC with tactile printer is the most flexible concept of the fax system since many different kind of hardware can already be in use by the user. In addition to existing hardware, a fax-modem, a sound unit and a parallel port are needed to upgrade the PC to a fax system. The sound unit enables sound and speech feedback.

Depending on available devices the user can choose which of the scenarios affects him most.

All scenarios include the ability to integrate additionally a regular fax machine connected via switchboard to fax system or PC. This enables the user to send ink printed faxes to himself and convert them into Braille or Moon and work on recognised graphics.

The scenarios including a personal computer, allows to connect a scanner to PC instead of using a standard fax machine. The software and hardware installation therefor has to be carried out by a third party.

Software modules

The choice of a graphical user interface respectively operating system might be counteracting the need for a simple user interface but as it will be detailed in the following does Windows 95 address a large variety of existing hardware components [4]. There is industrial standard hardware such as modems or printers and non-standardised hardware such as speech synthesisers available [1]. We develop a modular and flexible system with as many software interfaces as possible to make use of approved and reliable software and hardware components. Some of the software modules are professional software packages which require an application program with non-visual user interface in order to let blind users work with it.

The basic software modules of fax system are as shown in figure 1:

Image displaying how communication systems and hardware drivers are connected using TACFAX.
Figure 1: Software modules of TACFAX

Using standard modems, assisted by MS Windows 95 driver and configuration programs, the connection to the telephone network is build. The interface for telephony applications which is implemented in Windows 95 is called TAPI (Telephone Application Program Interface). For an unattended receiving of faxes we use MS Exchange which is also part of Windows 95 and includes the Mail Application Program Interface (MAPI). The communications between TACFAX and Microsoft Exchange is realized via this interface.

The OCR process is controlled by TACFAX through an state-of-the-art OCR toolkit. The OCR determines sections of graphics in document.

After a fax has been received, Microsoft Exchange has stored the image file on hard disk as temporary file. The OCR-process begins by loading this image file. After defining the set-up of OCR, all functions of the OCR process are executed automatically.

Through additional ‘language packages’ our fax system can read most European languages, even Albanian, Hungarian or Icelandic.

An image processing module includes routines to work on recognized graphics. The special graphical functions are used for loading, saving and printing of graphics. Furthermore, they offer the possibility to zoom and rotate graphics. The colour of the graphics (black/white, foreground, background) is changeable and is shown with different contrasts. In addition to this, noise is being removed if necessary.

The feedback of the fax system is realized via speech output. Therefor we use on one hand playback of sound files of format WAV and on the other hand synthetic speech [1]. We use the new standard of Microsoft ‘MS Speech’ for speech synthesis which address a large number of languages in Europe. In addition to this, SSIL will be supported.

The conversion of text in Braille or Moon is handled char by char trough TrueType fonts and Braille code definition files. A translation into Braille shorthand can be included later.

For printing tactile text and graphics we use the of Hewlett Packard developed PCL3 printer commands.

For the design of the user interface of the fax software we aim at three levels: beginner, advanced and expert. The user reaches these levels by experience.

The beginner uses the fax system as stand alone fax device which offers him the possibility to receive faxes and print them in Braille, Moon or ink print. Graphics will be printed automatically at the end of a document.

The advanced user not only receives faxes but may forward faxes and adapts the printing mode, according to his reading efforts. The advanced user identifies faxes by type and assigns a name (graphics processor).

The expert user can retrieve and reprint older faxes by name due to file-based handling of faxes. TACFAX therefor includes modules for connection with a word processor through HTML files and the direct networking feature of Windows 95. This means that the desktop PC ‘sees’ the file and the user selects a file for transfer on the desktop PC.

The non-visual user interface of the new fax device including auditory feedback will be tested by blind people in Marburg (Germany), Helsinki (Finland) and Lisbon (Portugal) in summer 1998.

Summary

The Fax system is developed in European project PRINT which is a consortium of members of diverse European states.

The new fax system parallels another quite common piece of equipment for blind people, the document reading system. The innovation of the fax system is the possibility to work on graphics and print them tactile on paper [5]. Because of this, the new fax system will be more readily acceptable for blind people. Getting Moon or Braille directly delivered from a standalone fax machine appears to be a straightforward approach for many users. For those users who want to work on the documents on a PC the choice of HTML as document encoding offers most alternatives.

The software modules which make up the fax system are based on standard commercial software. By applying industry standard software packages the fax system complies with many hardware requirements and is accessible to blind people.

REFERENCES

[1] Blenkhorn, P.: Producing a text-to-speech synthesizer for use by blind people, Extra-Ordinay-Human-Computer- Interaction, Cambridge University Press 1995: 307-314.

[2] Corvest, H.: A professional assessment of relief image publishing, New technologies in the education of the visually handicapped, Proceedings of the conference held in Paris June 12. 10-11, 1996: 71-76.

[3] Edman, P. K.: Tactile Graphics. New York: American Foundation for the Blind 1993.

[4]Edwards, A.D.N.: Graphical user interface and blind people, Computers for handicapped people, Proceeding of the 3rd international conference in Vienna July 7-9, 1992: 114-119.

[5] Lötsch, J.; Rödig, G.: Interactive tactile media in training visually handicapped people, New technologies in the education of the visually handicapped, Proceedings of the conference held in Paris June 10-11, 1996: 155-160.

[6] Möri, Daniel: Automatic recognition and execution of corrections on text documents (in German: Automatische Erkennung und Ausführung von Korrekturanweisungen in Textdokumenten), doctoral dissertation. Infix Press 1995.


Go to previous article. 
Go to next article. 
Return to 1998 Conference Table of Contents 
Return to Table of Proceedings


Reprinted with author(s) permission. Author(s) retain copyright.