2001 Conference Proceedings

Go to previous article 
Go to next article 
Return to 2001 Table of Contents


TELETEXT RECEIVER BY FINGER BRAILLE FOR DEAF BLIND

Yasuo Hoiuchi and Akira Ichikawa
1-33 Yayoi-cho, Inage-ku, Chiba-shi, Chiba, 263-8522, JAPAN
Email: hory@tj.chiba-u.ac.jp

Abstract

In this paper, we introduce a finger braille receiver for teletext broadcasting system. Finger braille is one of the communication methods for the deaf blind and it seems to be the medium most suited to real-time communication and for expressing the feelings of the speaker because of its prosody existing similarly to spoken languages. We assume that prosodic information is strongly needed to transform letters to finger braille. In this study, we analyzed the time structure of finger braille and found that it can be changed according to the structure and meaning of the sentences. Based on these results, we construct a prosody rule for time structure. The validity of the rule was confirmed in an output experiment. Finally, we made a prototype of teletext receiver by finger braille.

Finger Braille and Prosody

People who are both deaf and blind are called "deaf blind". They suffer much inconvenience in their everyday lives due to their handicap. In particular, the deaf blind with serious impairments cannot obtain sufficient information necessary for living, something which hearing and sighted people can do easily.

Typical communication methods for the deaf blind are (1) print-on-palm (tracing letters on the palm of the deaf blind), (2) tactile sign language and finger alphabet, (3) use of a "Bulista" which prints out braille on tape so the deaf blind can read it, and (4) finger braille using a Braille code. Of these methods, finger braille using a Braille code seems to be the best suited to real-time communication and for expressing the feelings of the speaker. In finger braille, the fingers of the deaf blind are regarded as keys of a brailler and a translator types the Braille codes on their fingers.

About 350 letters can be transmitted between a skilled deaf blind and a finger braille translator. As compared to oral transmission of 350 - 400 letters, finger braille is adequate for real-time communication. We believe that a real-time communication method such as speech or finger braille should convey not only linguistic information but also paralinguistic and nonlinguistic information. Spoken languages employ all types of prosody, which enhance the real-time understanding of the utterances. Here, we assume that finger braille using a Braille code as a real-time communication method also contain such information (we call it the prosody of finger braille). By examining the prosodic information of spoken languages, we can determine such factors as the sentence structure, sentence type (e.g., question, declaration, etc.), and prominence. It is suggested that there is similar prosodic information in finger braille.

Equipment for finger braille has recently been proposed (Hoshino et al., 1996; Hoshino et al., 1998; Nakashi & Shida, 1998). However, no consideration has been given to the prosody. In order to realize a teletext receiver, we developed a finger braille output unit which can transmit not only braille codes but also the prosody, especially its timing structure, such that the deaf blind can understand finger braille well. To accomplish this, we first analyze the timing structure of finger braille and a prosody rule for finger braille is proposed. Subjective experiments are performed to evaluate the rule.

Analysis of the Timing Structure of Finger Braille

Analysis of Prominence Word

To examine the time structure of finger braille, we have developed a new instrument for measuring the prosody of finger braille. Force-sensitive resistors are adopted to detect finger pressure. The output from the six sensors (three for each hand such as in the case of a brailler) is input to a PC every 10 milliseconds.

The subject has much experience as a finger braille translator. In the recording, the subject was asked to answer questions using the same sentence. The answers by the translators to all the questions addressed are the same, however, the positions of prominent words change according to the particular question. For example, the answer is "3 jini chibaekino higashiguchidesu (Meet me at the east exit of Chiba station at 3 o'clock), the following three questions were addressed to the subject. The question 1 is "nanjini chibaekino higashiguchidesuka ? (At what time will we meet at the east exit of Chiba station?)", the question 2 is "3 jini donoekino higashiguchidesuka ? (At which station will we meet at 3 o'clock?)" and the question 3 is "3 jini chibaekino dokodesuka ? (At which exit of Chiba station will we meet at 3 o'clock ?)".

The duration between the onset of pressure of one typed finger code and the onset of the next one is defined as the duration of the typed code. The results show that the duration of the last code of each phrase is longer than that of other codes (shown in 91 % of all recording) and that the duration of the last code of the prominent word and the code just before the prominent word are appreciably longer than the others (shown in 73 % of all recording). These results indicate that the long duration clarifies the boundary of each phrase or prominent word.

Analysis of Ambiguous Sentences

In the second recording, the subject was asked to type the ambiguous sentences, (sentences which have two meanings) so as to discriminate their meanings (same code sequences but different meaning). For example, an ambiguous sentence is "Wakaiotokoto onnaga aruiteiru (young man and woman are walking)" where this sentence does not give sufficient information to distinguish whether the word `wakai (young)' applies to only the man or both the man and the woman. However, in oral transmission, the meaning can be distinguished from the change of pitch, power and timing structure of the sound (prosody of spoken language). We assume that the timing structure of finger braille has the same function. Seven different sentences that each has two meanings were recorded. During the recording, the subject consciously types the sentences as if he tries to convey two different meanings to the deaf blind. For each meaning, the recording was performed twice.

The result of the first recording (about Prominence Word) suggests that a short duration indicates a strong combination between two codes. Hence we made a "prosodic tree" by combining the codes in order of short duration. The resulting trees represent the semantic structure of the recorded sentences and two trees with different meanings are discreminated well. It is suggested that the timing structure of finger braille is affected by not only the structure of sentences but also the meaning of the sentences. These findings support our assumption.

Analysis of Paragraph

In the third recording, the three translators were asked to type short paragraphs from a news program in order to determine the parameter of the prosody rule. The subjects listen to the news and type the paragraph.

Table 1 shows the average duration of the last code of phrases and sentences, and some special codes. Table 1. Average values of duration by codes Types of the code Duration (milliseconds) Last code of phrase 790 Last code of sentence 697 Code for voiced syllable 343 Code for palatalized syllable 357 Code to change character set 587 Others 377

For example, the code which acts to change an unvoiced consonant into a voiced consonant, has a short duration, while the code which changes the coding system (e.g. changing into the numerical mode) has a long duration. The result indicates that the length of duration has much to do with the function of the special codes. If the deaf blind fail to read the vocalization code, they will misread a following code only. However, if they skip the numberization code, it is possible that more than two codes are not transformed to number and likely misread. It causes a serious influence on understanding of the sentence. Therefore, duration of transform codes became longer, so the codes will no be skipped.

Prosody Rule for Timing Structure

From the above results, we derive a rule to model the prosody information of finger braille. The structure of the sentence and the type of the code determine a duration of each braille code. The code is analyzed in advance whether it is the end of a phrase or a sentence, and whether it is a special code. Each code is given the average values as its duration.

Output Experiments

An experiment has been performed to evaluate the effectiveness of the prosody rule. We examined whether the deaf blind have a better understanding when prosody information is added to finger braille output. We have developed a new instrument for output of finger braille. It is available to control the time structure of output by PC.

The subject was a deaf blind who uses the finger braille as her major communication means. Before the experiment, there was a rehearsal. The subject could read all the sentences both with and without prosody.

In the experiment, to compare two outputs effectively, the parameter was set as half the recorded time, so the output speed became twice the recorded time. Four essays about animal lives were output. One essay had 450 - 500 letters and consisted of three paragraphs. Two essays had prosodic information and two had no prosodic information. There were 10 questions concerning each essay, so 20 questions were prepared for each outputs. A finger braille translator typed the questions, and the subject answered orally. The questions were repeated until the subject understood them completely.

Table 2 shows the results of the experiment. Table 2: Results of the experiment Output Percentage of correct answers With prosody 85 % (17 correct answers) Without prosody 50 % (10 correct answers) The subject exhibited a better understanding of the output with the prosody rule. The subject felt that the output by prosody were more natural and understandable as to the timing structure of sentences. The similar results were shown in a study of prosody of spoken languages (Kitahara et al., 1987). The result confirms the validness of the prosody rule.

Finger Braille Receiver for Teletext Broadcasting System

Our output system based on the prosody rule can be a real-time communication method that can help the deaf blind to obtain information and communicate effectively with others. We developed a finger braille receiver for Japanese teletext broadcasting system which help the deaf blind to use current mass media. Although similar system has recently been proposed (Sakai et al., 1998), no consideration has been given to the prosody.

In our system, a PC receives the teletext and braille codes are output according to the prosody rule. The outline of the process is (1) receiving teletext, (2) converting kanji text into kana character, (3) converting kana character into braille code, (4) carrying out morphological analysis and syntactic analysis of the teletext sentence, and (5) applying the prosody rule.

Conclusions

In this paper, we introduce the prosody rule for finger braille based on the analysis of actual data and the effectiveness of the rule was shown by the output experiment. Finally, we made a teletext receiver by finger braille.

Using the finger braille output instrument, the deaf blind may be able to talk to other deaf blind on the telephone. Furthermore, hearing and sighted people may be able to talk to them on the telephone by using a converter for braille and speech or a keyboard.

We are currently designing a rule to express the feelings of the speaker using finger braille output. We also focus on finger pressure in finger braille.

Acknowledgements

The authors are grateful to Mr. Uchiyama, and Mr. Wada of Double R&D Co., Ltd. for making the finger braille instruments. We also wish to thank the volunteer subjects for their assistance. References

Hoshino et al., 1996; An Experimental Communication Tool by Finger Braille, In Proceedings of the 22nd Sensory Substitution Symposium (pp. 107-110), Tokyo, Sensory Substitution Symposium.

Hoshino et al., 1998; Basic Study of Optimum Stimulus Mode for Finger Braille Display, The Transactions of the Institute of Electronics, Information and Communication Engineers A 81(9), 1273-1279.

Nakashi, M. & Shida, K. 1996; Finger Braille based on electrical stimulation for the overlap handicapped person. In Proceedings of the 24th Sensory Substitution Symposium (pp. 121-124), Tokyo, Sensory Substitution Symposium.

Kitahara et al., 1987; Study of Prosodic Effect on Acceptance of Spoken Language, In National Convention Record, 1987 (pp. 6-17), Yokohama, The Institute of Electronics, Information and Communication Engineers.

Sakai et al. 1998; Study of New Passive Tactile Braille Transmission Methods - Development of Braille Transmission Equipment for Teletext Broadcasting-, The Journal of the Institute of Image Information and Television Engineers, 52(4), 512-517.


Go to previous article 
Go to next article 
Return to 2001 Table of Contents 
Return to Table of Proceedings


Reprinted with author(s) permission. Author(s) retain copyright.