2000 Conference Proceedings
Go to previous article
Go to next article
Return to 2000 Table of Contents
Concentration Game: an Audio Adaptation for the blind
Patrick Roth (1), Lori Petrucci (1), André
Assimacopoulos (2), Thierry Pun (1)
(1) Computer Science Dpt.,
University of Geneva
24, rue du Général Dufour
CH - 1211 Geneva 4, Switzerland
E-mail: Patrick.Roth@cui.unige.ch
(2) UCBA/SZB - Swiss Central Of and For the Blind
Schützengasse 4
CH - 9000 St.-Gallen, Switzerland
Abstract In this paper, we describe an auditory adaptation
for blind people of the well known Concentration Game. Our
approach is characterized by two main aspects : 1) the use of a
3D virtual sound space to represent the game elements; 2) the
inclusion of a training modality by which the blind user can
easily be familiarized with the audio space. An evaluation of
the game was made with four participants. During this
evaluation process, we found that blind users play with the
game differently according to the nature of their blindness as
well as to their musical training.
Introduction
Computer games are very useful tools to help teach concepts
such as spatial organization, mathematics, etc. This is
especially true for blind pupils and adults. In their case, in
addition to providing an easy familiarization with the
computer, the games can be tailored to help them acquire
particular skills. Unfortunately, as with all software working
with graphical user interfaces (GUI), most games use vision as
the principal communication channel. A certain category of
games however, such as those based on a square layout (i.e. tic
tac toe, chess, ship battle war), have been accessible to the
blind community for a long time. The reason is that they were
developed during the DOS era [8]; due to the simplicity of the
interface, most DOS softwares were fully accessible to the
blind. Recently, triggered by the advent of multimedia
computing, various investigations have been made regarding the
use of verbal and non-verbal cues to convey visual information
[6,9]. In this context, we believe that sonifying games for the
blind is essential to help them learn how to cope with
computers, as well as to teach them some spatial concepts using
the auditory sense. In this paper, we therefore present our
work on the sonification of the well known concentration game.
Despite the fact that we now work on the sonification of other
games, we started with the concentration game since it was a
direct extension of our earlier work on GUI sonification for
blind users [7,10] and at the same time a perfect learning tool
(for geometry concepts in our case).
The Sonic Concentration Game
Sonic concentration game rules The rules we adopted are similar
to those of the traditional concentration game. The game
contains three different levels, each level offering 4 pairs of
geometrical shapes to be matched. These shapes, a straight
line, a square, a circle, a rectangle, are first sonified (see
Sonification Principles below) and then randomly hidden
"behind" a rectangular array comprising eight cells. The goal
is to associate the 4 pairs of shapes together, after the
player has heard all the 8 auditory cues. For instance, if our
array is configured as {square, rectangle, rectangle, circle,
square, triangle, circle, triangle}, then the expected answer
will be {(1,5),(2,3),(4,7),(6,8)}. Sonification principles In
order to represent a geometrical shape in terms of a series of
sounds, we investigated an approach based on sound localization
in a 3D sound space using headphones [2,12]. In this approach,
a shape is rendered by a moving sound that is acoustically
drawn within a bidimensional sound space. This bidimensional
sound space can be either horizontal or vertical according to
the player's whish. For instance, if the shape to be rendered
is a square in a vertical plane, the player will hear in that
plane the sound moving horizontally from the upper left to the
upper right, descend vertically, move horizontally from the
bottom right to the bottom left and finally ascend to reach the
initial position. Concerning auditory perception, much research
points out to the difficulty in localizing a sound on the
azimutal plan as well as in differentiating the front from the
back [12]. Therefore, in order to deal with these problems, a
number of functionnalities were added such as : a reinforcement
of the elevation rendering when using vertical plane, by means
of a frequency variation [11]. Thus, a decrease in highness
will be perceived as a frequency decrease; the use of the
Doppler effect [3] to enhance the front/back differences; the
possibility for the player to choose their sounds, as either
melodic (represented by a sinusoidal timbre) or ecological
(such as the sound of an helicopter); a particular coding of
line junctions for patterns composed of several lines (e.g.
square, triangle) by using alarms such as a "beep". Learning
modality Brigitte Assénat wrote in her paper [1]:
"A child does not play to learn, but learns because he plays.
This is not true of a child with visual disabilities since he
must learn how to play before playing".
This emphasizes the fact that it is crucial to embed a
learning session in each game that we create for the blind and
visually impaired. The training stage that we developed is
based on an action-perception loop in which the action is
caused by the finger movement and the perception by means of
two channels: tactile and auditory. The use of tactile
perception therefore allows for a better learnability of the
auditory environment.
Concretely, we created for all the geometrical shapes we used
their tactile representations using a specialized thermal
printer that produces output on swell paper [5]. The swell
paper outputs were then placed on a graphic tablet so that the
user was able to follow the shape edges while listening to
their audio rendering. A graphical tablet for capturing finger
movements was preferred to a touch sensitive screen for two
reasons: first, it provides absolute coordinates; secondly,
using a horizontal pointing device causes less fatigue than
using a vertical touch sensitive screen [4]. For the auditory
output, we mapped the graphical tablet into one of the two
auditory planes mentioned earlier. In summary, during the
training stage, the blind user explores the tactile
representation with his/her finger and the system responds with
an audio feedback related to the finger position on the
graphical tablet.
Implementation
The implementation and evaluation of the concentration game was
made on a Pentium PII equipped with a SoundBlaster Live sound
card. For the 3D sound environment, we used the DirectSound3D
Library available from Microsoft. For the learning modality, we
took a WACOM Intuos A4 graphical tablet.
Evaluation
We report here a first series of evaluations of the game with
blind adults. We choose to first test the game with adults
rather than children because blind adults have a longer
experience in using geometrical shape representations, and they
can provide more useful feedback to help in the design of the
game. Participants and scenario The concentration game was
tested with four blind people aged 20-35. Two of them were
totally blind since birth, the third since the age of 8 and the
fourth since the age of 13. Three participants have previous
musical experience. After a short oral explanation of the
rules, the participants got accustomed with the auditory
environment by using the learning modality for a period that
did not exceed 20 minutes. They then started to play the game.
Depending on the game level, the following shapes were used :
beginner level : horizontal line, vertical line, diagonal line
starting at the upper left corner, diagonal line starting at
the bottom left corner; medium level : diagonal line starting
at the bottom left corner, square, triangle, circle; expert
level : square, rectangle, circle, oval. At each level, the
participants were asked to perform the following two tasks : to
associate the corresponding auditory cues; to determine to
which geometrical shapes the auditory cues corresponded. At the
end of the experiment, we asked the users to comment on the
difficulties they encountered.
Results and discussion
Concerning the need for a learning session, we observed that
this stage was much more useful for the congenitally blind than
for the other users. This might come from the fact that people
blind since birth were only able to learn geometry using the
haptic sense; their mental representation of geometrical shapes
is therefore not as sharp as with late blind users. Generally,
all the participants were able to perform the two tasks. The
only problem occurred with the differentiation between the oval
and circular shapes. We noticed that people blind since birth
and with prior musical training found a way that permitted them
to make this differentiation. Essentially, they reversed the
training stage: rather than exploring the tactile shapes and
listening to the output, they listened to the audio rendering
and tried to reconstruct the tactile representation of the oval
and circle.
Regarding the use of the bidimensional sound space, most users
preferred to work with the horizontal plane. The reason
pertains to the fact that in their everyday life, blind people
are accustomed to horizontal displacements. In our case, we
however still had to retain the possibility to use the vertical
plane since the front/back discrimination offered by our
program was not always satisfactory.
All participants found that the additional coding of the line
junction was very useful for the following two reasons. First,
it permitted to anticipate the changes of the moving sound
direction; secondly, it allowed to count the lines that the
shape included.
Regarding the need for an additional frequency variation,
people with musical experience directly took advantage of this
feature. This was not the case for the non-musicians who had to
go through a learning process in order to assimilate the
frequency/highness representation. For this task, non-musicians
preferred an ecological timbre for the audio rendering.
As a final comment, all users believed that this game could be
very useful for young pupils in their education.
Conclusions
Based on the results obtained during the evaluation process, we
can conclude that : the participants recognized and were able
to match all the shapes that we proposed, except for the oval
and the circle. After further training on these shapes only,
one user was still not able to distinguish between them; the
training task was more useful for people blind since birth;
blind people with musical experience had more facility to play
with this game. The next step will consist of the evaluation of
the game with young pupils. We think however that this game has
to be played by pupils who already have a good knowledge of
spatial geometry. To go further in geometrical training we will
include a wider variety of common polygonal shapes.
Further work is devoted to the adaptation of other games, as
well as to the use of different entry devices such as joystick
and force-feedback mouse. We are also currently finalizing the
design of an audio exploration tool for graphics, which will
allow the acoustic analysis of simple drawings as well as to
create one's own pictures.
ACKNOWLEDGMENTS
This project is financed by the Swiss Priority Program in
Information and Communication Structures, by the Swiss Central
Union Of and For the Blind and the Association pour le Bien des
Aveugles. The authors are grateful to M.-P. Assimacopoulos, A.
Barrillier, A. Bullinger, J. Conti for their help in the design
and evaluation of the prototype.
References
[1] Assena, B., Dupuis, H., Gregoire, L., Apprenez-moi à
jouer: Aux parents d'enfants déficients visuels de 2
à 5 ans, Info-Doc, 8, 3, 1er trim 98.
[2] Blauert, J. Spatial Hearing, MIT Press, MA, 1983.
[3] Gill, T. P., The Doppler Effect, Logos Press, 1965.
[4] Gokturk, M., Sibert, J. L., An Analysis of the Index
Finger as a Pointing Device, CHI99 Proceedings, May 1999, pp.
286-287.
[5] Lange, M., Design of Tactile Graphics, HCI99 Proceedings
Aug. 1999, pp. 990-994.
[6] Lumbreras, M., Sanchez, J., Interactive 3D Sound
Hyperstories for Blind Children, CHI99 Proceedings, May 1999,
pp. 318-325.
[7] Petrucci, L.S., Roth, P., Assimacopoulos, A., Pun, T., An
Audio Browser in Increasing Access to World Wide Web Sites for
Blind and Visually Impaired Computer Users, HCI99 Proceedings
Aug. 1999, pp. 995-997.
[8] Resources for Parents & Teachers of Blind Kids,
http://www.az.com/~dday/Textonly.html
[9] Raman, T.V., Conversational Gestures For Direct
Manipulation On The Audio Desktop, ASSETS'98 Proceedings, ACM
Press.
[10] Roth, P., Petrucci, P., Assimacopoulos, A., Pun, T.,
AB-Web: Active audio browser for visually impaired and blind
users, ICAD'98 Proceedings, Nov. 1998.
[11] Shimizu, M., Itoh, K., Nakazawa, T., Pattern
Representation System using Movement Sense of Localized Sound,
HCI99 Proceedings Aug. 1999, pp. 990-994.
[12] Wightman, F.L., & Kistler, D.J. (1989b) "Headphone
simulation of free-field listening II: Psychophysical
validation, "Journal of the Acoustical Society of America,
85(2), pp. 868-878.
Go to previous article
Go to next article
Return to 2000 Table of Contents
Return to Table of
Proceedings
Reprinted with author(s) permission. Author(s) retain copyright.