2000 Conference Proceedings

Go to previous article 
Go to next article 
Return to 2000 Table of Contents


Concentration Game: an Audio Adaptation for the blind

Patrick Roth (1), Lori Petrucci (1), André Assimacopoulos (2), Thierry Pun (1)
(1) Computer Science Dpt.,
University of Geneva
24, rue du Général Dufour
CH - 1211 Geneva 4, Switzerland
E-mail: Patrick.Roth@cui.unige.ch 

(2) UCBA/SZB - Swiss Central Of and For the Blind
Schützengasse 4
CH - 9000 St.-Gallen, Switzerland

Abstract In this paper, we describe an auditory adaptation for blind people of the well known Concentration Game. Our approach is characterized by two main aspects : 1) the use of a 3D virtual sound space to represent the game elements; 2) the inclusion of a training modality by which the blind user can easily be familiarized with the audio space. An evaluation of the game was made with four participants. During this evaluation process, we found that blind users play with the game differently according to the nature of their blindness as well as to their musical training.

Introduction

Computer games are very useful tools to help teach concepts such as spatial organization, mathematics, etc. This is especially true for blind pupils and adults. In their case, in addition to providing an easy familiarization with the computer, the games can be tailored to help them acquire particular skills. Unfortunately, as with all software working with graphical user interfaces (GUI), most games use vision as the principal communication channel. A certain category of games however, such as those based on a square layout (i.e. tic tac toe, chess, ship battle war), have been accessible to the blind community for a long time. The reason is that they were developed during the DOS era [8]; due to the simplicity of the interface, most DOS softwares were fully accessible to the blind. Recently, triggered by the advent of multimedia computing, various investigations have been made regarding the use of verbal and non-verbal cues to convey visual information [6,9]. In this context, we believe that sonifying games for the blind is essential to help them learn how to cope with computers, as well as to teach them some spatial concepts using the auditory sense. In this paper, we therefore present our work on the sonification of the well known concentration game. Despite the fact that we now work on the sonification of other games, we started with the concentration game since it was a direct extension of our earlier work on GUI sonification for blind users [7,10] and at the same time a perfect learning tool (for geometry concepts in our case).

The Sonic Concentration Game

Sonic concentration game rules The rules we adopted are similar to those of the traditional concentration game. The game contains three different levels, each level offering 4 pairs of geometrical shapes to be matched. These shapes, a straight line, a square, a circle, a rectangle, are first sonified (see Sonification Principles below) and then randomly hidden "behind" a rectangular array comprising eight cells. The goal is to associate the 4 pairs of shapes together, after the player has heard all the 8 auditory cues. For instance, if our array is configured as {square, rectangle, rectangle, circle, square, triangle, circle, triangle}, then the expected answer will be {(1,5),(2,3),(4,7),(6,8)}. Sonification principles In order to represent a geometrical shape in terms of a series of sounds, we investigated an approach based on sound localization in a 3D sound space using headphones [2,12]. In this approach, a shape is rendered by a moving sound that is acoustically drawn within a bidimensional sound space. This bidimensional sound space can be either horizontal or vertical according to the player's whish. For instance, if the shape to be rendered is a square in a vertical plane, the player will hear in that plane the sound moving horizontally from the upper left to the upper right, descend vertically, move horizontally from the bottom right to the bottom left and finally ascend to reach the initial position. Concerning auditory perception, much research points out to the difficulty in localizing a sound on the azimutal plan as well as in differentiating the front from the back [12]. Therefore, in order to deal with these problems, a number of functionnalities were added such as : a reinforcement of the elevation rendering when using vertical plane, by means of a frequency variation [11]. Thus, a decrease in highness will be perceived as a frequency decrease; the use of the Doppler effect [3] to enhance the front/back differences; the possibility for the player to choose their sounds, as either melodic (represented by a sinusoidal timbre) or ecological (such as the sound of an helicopter); a particular coding of line junctions for patterns composed of several lines (e.g. square, triangle) by using alarms such as a "beep". Learning modality Brigitte Assénat wrote in her paper [1]:

"A child does not play to learn, but learns because he plays. This is not true of a child with visual disabilities since he must learn how to play before playing".

This emphasizes the fact that it is crucial to embed a learning session in each game that we create for the blind and visually impaired. The training stage that we developed is based on an action-perception loop in which the action is caused by the finger movement and the perception by means of two channels: tactile and auditory. The use of tactile perception therefore allows for a better learnability of the auditory environment.

Concretely, we created for all the geometrical shapes we used their tactile representations using a specialized thermal printer that produces output on swell paper [5]. The swell paper outputs were then placed on a graphic tablet so that the user was able to follow the shape edges while listening to their audio rendering. A graphical tablet for capturing finger movements was preferred to a touch sensitive screen for two reasons: first, it provides absolute coordinates; secondly, using a horizontal pointing device causes less fatigue than using a vertical touch sensitive screen [4]. For the auditory output, we mapped the graphical tablet into one of the two auditory planes mentioned earlier. In summary, during the training stage, the blind user explores the tactile representation with his/her finger and the system responds with an audio feedback related to the finger position on the graphical tablet.

Implementation

The implementation and evaluation of the concentration game was made on a Pentium PII equipped with a SoundBlaster Live sound card. For the 3D sound environment, we used the DirectSound3D Library available from Microsoft. For the learning modality, we took a WACOM Intuos A4 graphical tablet.

Evaluation

We report here a first series of evaluations of the game with blind adults. We choose to first test the game with adults rather than children because blind adults have a longer experience in using geometrical shape representations, and they can provide more useful feedback to help in the design of the game. Participants and scenario The concentration game was tested with four blind people aged 20-35. Two of them were totally blind since birth, the third since the age of 8 and the fourth since the age of 13. Three participants have previous musical experience. After a short oral explanation of the rules, the participants got accustomed with the auditory environment by using the learning modality for a period that did not exceed 20 minutes. They then started to play the game. Depending on the game level, the following shapes were used : beginner level : horizontal line, vertical line, diagonal line starting at the upper left corner, diagonal line starting at the bottom left corner; medium level : diagonal line starting at the bottom left corner, square, triangle, circle; expert level : square, rectangle, circle, oval. At each level, the participants were asked to perform the following two tasks : to associate the corresponding auditory cues; to determine to which geometrical shapes the auditory cues corresponded. At the end of the experiment, we asked the users to comment on the difficulties they encountered.

Results and discussion

Concerning the need for a learning session, we observed that this stage was much more useful for the congenitally blind than for the other users. This might come from the fact that people blind since birth were only able to learn geometry using the haptic sense; their mental representation of geometrical shapes is therefore not as sharp as with late blind users. Generally, all the participants were able to perform the two tasks. The only problem occurred with the differentiation between the oval and circular shapes. We noticed that people blind since birth and with prior musical training found a way that permitted them to make this differentiation. Essentially, they reversed the training stage: rather than exploring the tactile shapes and listening to the output, they listened to the audio rendering and tried to reconstruct the tactile representation of the oval and circle.

Regarding the use of the bidimensional sound space, most users preferred to work with the horizontal plane. The reason pertains to the fact that in their everyday life, blind people are accustomed to horizontal displacements. In our case, we however still had to retain the possibility to use the vertical plane since the front/back discrimination offered by our program was not always satisfactory.

All participants found that the additional coding of the line junction was very useful for the following two reasons. First, it permitted to anticipate the changes of the moving sound direction; secondly, it allowed to count the lines that the shape included.

Regarding the need for an additional frequency variation, people with musical experience directly took advantage of this feature. This was not the case for the non-musicians who had to go through a learning process in order to assimilate the frequency/highness representation. For this task, non-musicians preferred an ecological timbre for the audio rendering.

As a final comment, all users believed that this game could be very useful for young pupils in their education.

Conclusions

Based on the results obtained during the evaluation process, we can conclude that : the participants recognized and were able to match all the shapes that we proposed, except for the oval and the circle. After further training on these shapes only, one user was still not able to distinguish between them; the training task was more useful for people blind since birth; blind people with musical experience had more facility to play with this game. The next step will consist of the evaluation of the game with young pupils. We think however that this game has to be played by pupils who already have a good knowledge of spatial geometry. To go further in geometrical training we will include a wider variety of common polygonal shapes.

Further work is devoted to the adaptation of other games, as well as to the use of different entry devices such as joystick and force-feedback mouse. We are also currently finalizing the design of an audio exploration tool for graphics, which will allow the acoustic analysis of simple drawings as well as to create one's own pictures.

ACKNOWLEDGMENTS

This project is financed by the Swiss Priority Program in Information and Communication Structures, by the Swiss Central Union Of and For the Blind and the Association pour le Bien des Aveugles. The authors are grateful to M.-P. Assimacopoulos, A. Barrillier, A. Bullinger, J. Conti for their help in the design and evaluation of the prototype.

References

[1] Assena, B., Dupuis, H., Gregoire, L., Apprenez-moi à jouer: Aux parents d'enfants déficients visuels de 2 à 5 ans, Info-Doc, 8, 3, 1er trim 98.

[2] Blauert, J. Spatial Hearing, MIT Press, MA, 1983.

[3] Gill, T. P., The Doppler Effect, Logos Press, 1965.

[4] Gokturk, M., Sibert, J. L., An Analysis of the Index Finger as a Pointing Device, CHI99 Proceedings, May 1999, pp. 286-287.

[5] Lange, M., Design of Tactile Graphics, HCI99 Proceedings Aug. 1999, pp. 990-994.

[6] Lumbreras, M., Sanchez, J., Interactive 3D Sound Hyperstories for Blind Children, CHI99 Proceedings, May 1999, pp. 318-325.

[7] Petrucci, L.S., Roth, P., Assimacopoulos, A., Pun, T., An Audio Browser in Increasing Access to World Wide Web Sites for Blind and Visually Impaired Computer Users, HCI99 Proceedings Aug. 1999, pp. 995-997.

[8] Resources for Parents & Teachers of Blind Kids, http://www.az.com/~dday/Textonly.html

[9] Raman, T.V., Conversational Gestures For Direct Manipulation On The Audio Desktop, ASSETS'98 Proceedings, ACM Press.

[10] Roth, P., Petrucci, P., Assimacopoulos, A., Pun, T., AB-Web: Active audio browser for visually impaired and blind users, ICAD'98 Proceedings, Nov. 1998.

[11] Shimizu, M., Itoh, K., Nakazawa, T., Pattern Representation System using Movement Sense of Localized Sound, HCI99 Proceedings Aug. 1999, pp. 990-994.

[12] Wightman, F.L., & Kistler, D.J. (1989b) "Headphone simulation of free-field listening II: Psychophysical validation, "Journal of the Acoustical Society of America, 85(2), pp. 868-878.


Go to previous article 
Go to next article 
Return to 2000 Table of Contents 
Return to Table of Proceedings


Reprinted with author(s) permission. Author(s) retain copyright.