2000 Conference Proceedings

Go to previous article 
Go to next article 
Return to 2000 Table of Contents

Do you see what I see? - To be, or not to be, equal when perceiving the digital universe

Rui Raposo (raposo@ca.ua.pt)
Department of Communication and Arts
University of Aveiro - Portugal

Abstract

A blank sheet of paper seems to help us in our reflective actions. Nothing exists before it, there isn’t a need to follow anyone’s steps or ideas, we hold before us a container for our own ideas possible of description in words, phrases, or drawings. This is how I see that blank piece of paper, how do you see it? Your perception probably might not be so different from my own, but it is still different. This has everything to do with what perception is all about. It definitely differs in many aspects between two individuals, but it probably comes across as similar to a larger numbers of observers.

Ergonomic issues come as essential when conceiving multimedia applications. Discussions on colours, images, icons, language (verbal and non verbal) are often held aiming at a total, or at least, nearly total satisfaction of the individuals needs and wills when using the program. The main question here is to what extent are we satisfying a majority and leaving out a minority. To what point is it known, and scientifically proven that people with severe mental retardation share the same response to, for instance, the colour red as someone without any mental retardation what so ever? If visual perception and interpretation does differ between individuals with and without severe mental retardation, there is much to be learnt towards the correct conception of multimedia applications for people with this kind of special need.

I will be presenting a preview of some of the work I’m developing in my post graduate studies on visualisation of the digital world by people with severe mental retardation.

Context

The development of our society, the so-called information society, guides us at a first level, to its comprehension as an open system in order to understand the importance of the accessibility of new technologies for all. The approach to this matter while exploring the analogy to biologically natured open systems enables us to identify various pieces of a greater puzzle. We are able to identify human community occupied spaces as the environment. The human beings as the subsystems which co-ordinate their activities among themselves in order to achieve solid points of equilibrium that if altered, due to the emergence of new inputs, contain the ability to readjust to new levels of equilibrium favourable to the survival of the system. Finally it is possible to identify inputs and outputs of the system, which in their core are information and communication actions.

Through the analysis of society in this metaphorical manner it is possible to conclude that the partial or total non-supply of inputs, in this case information, to an individual which belongs to the system will lead him or her to an inefficient playing of his role as a sub-system. Due to this, the citizen will not contribute with outputs which will benefit the other sub-systems, the global system – in this case the information society – and he will possibly influence in a negative way, his own functionality.

There are various bridges that can be built between the open system metaphor and the issue of accessibility of people with special needs to multimedia applications conceived correctly for them. A correct comprehension of the relationship between these two margins may be achieved when taken into mind the following chain reaction, which begins at the individual/sub-system level and evolves onto a wider system level:

The individual human being when interacting with communication and information new technologies of correctly adapted to his or her needs discovers a new media with purposes, which may go from entertainment and leisure areas to the simple expression of ideas and feelings. The discovery of this new way of having fun and communicating with others may help the improvement of his or her auto-esteem and the feeling of well being. This improvement of equilibrium at the individual level and the emission of positive outputs, interpreted by other systems as positive inputs, will contribute to the improvement of the relationship between the individual and the other sub-systems as well as the individual’s active role as a society member. If we take into example people with special needs, we are able to identify family and friends as the sub-systems, which will be directly influenced by this gain of effectiveness by their loved ones and friends. The last stage of equilibrium looked for is the global systems, in this case the information society. The improvement of its member’s efficiency enables the global system to experience a close to perfect equilibrium existence. Through this it becomes easy to identify the importance of the need for the correct adapting of new technologies for people with special needs. However, this adapting process is not at all a straightforward one. There is a need to deconstruct the media in order to understand each one of its components separately. It is not a case of simply just adapting the hardware or the software. The contents of the application and the manner in which they are presented must also be carefully analysed.

My work has mainly to do with the study of visual stimuli used in the conception of multimedia applications for people with sever mental retardation. I emphasise on the colours that are just one of the many pieces of the puzzle. One of the main problems one might come across while studying visual stimuli and their effect on people with mental retardation has to do with the scarce existence of means and tools capable of helping the analysis of this sort of stimuli. If we add to this the fact that the existing tools have nothing, what so ever, to do with new technologies, the whole study becomes more and more complicated. The present exercises, which comprehend the use of, coloured cards, building blocks and paper supported questionnaires, lack the link towards modern society and the use of computers in research studies. In a society in which we are constantly made aware of the need to co-operate in the implementation of a horizontally levelled information society, new technologies used in communication actions must be subjected to an analysis phase. Its content must be deconstructed and adapted to the real needs of people who are sometimes left behind or set aside from society’s progress plans. However, these two analysis actions – deconstruction and adaptation – are not easy tasks to undertake.

Our first challenges to overcome are not the negative attitude of people that will use the computer or electronic device, but instead of the people who will, in some way, help them use it. In some cases the people who decide the inclusion, or not, of new technologies in the working or leisure activities of schools and institutions are the first to close the door to progress. Fortunately they are a species seconds away from extinction. The second problem faced is the lack of computer applications correctly conceived for people with special needs. This is yet another problem that gradually has evolved into a better scenario thanks to organisations such as Closing the Gap and companies such as Intellitools. There is no point in bringing the computer into the classroom if it does not present itself as a improvement to the education program or if it just something left lying around and never used. These new technologies must be adapted to help the user focus on the exercise or the message and not the machine. In order for this to work we must put the user in the centre of our creativity universe working towards what he or she wants and needs. A handicap, on it’s own, may not be a barrier to the adapting of an individual to an environment or situation. As long as that same environment is related to or adapted to his or her identifiable characteristics, there is no reason not to believe that an individual can not undertake various tasks with acceptable success [Sternberg & Spear85].

The goal of contributing to the task of deconstructing the information and helping to identify optimised information for the conception of multimedia applications for people with severe mental retardation led me to the design of a interactive system for research on visual stimuli. Initially the focus point of the research is set to be colour, but the plasticity of the system will allow in the future the crossing into other research subjects. These might be other types of visual stimuli (imagery, animation), sound stimuli or the association of two or more stimuli and the study of the subsequent response from the user. The interactive system has been called Tic-Tac-Touch.

Tic-Tac-Touch

How do you create a computer application to be used by people with severe mental retardation, which will allow you to gather information on their preferences towards colours? This is a pretty complicated question to answer. Maybe the first thing to do is observe the target audience we intend to reach out and touch and find out what "makes them tick". Occupational therapy is a great reference on this issue. Years of work have helped professionals in occupational therapy to identify activities suitable for people with specific special needs. Why not ask them? This is the point where trans-disciplinary efforts start to take place. Through several conversations with occupational therapists from PACP (Portuguese Association for Cerebral Palsy) we were able to identify finger painting as a common activity which might be successful in a computer-mediated context. This activity seemed to present a world of possibilities in terms of interaction and information on colour preferences. The next stage was to convert finger painting into a metaphor capable of being translated onto a computer screen and through a computer based system. The spawning of this metaphor may be divided into three main steps [Erickson96]:

Functional definition

The first step was to identify the market and budget available technological resources for this study. There was also a careful study concerning the technological limits and the optimised use of those same resources.

Identification of the habits and needs

Hypothetical patterns of possible habits and needs were drawn-up including the changes derived from the shifting of an activity from the real world onto an abstract touchscreen environment.

Metaphor generating The final step was to translate into a touchscreen dependent exercise the real life finger painting exercise. The details of this translation and further details on the finger painting program and the Tic-Tac-Touch interactive system are described through out this paper.

This digital finger painting application presents itself as a junction of the research effort on multimedia applications for people with mental retardation and the conception of these sort of programs. This means that in spite of the research aspect of the application, it is still usable in non-research contexts such a leisure and creative activity in occupational therapy sessions. The research aspect of the activity must not be put aside for it might give us some very important information on the colour preference patterns in users with severe mental retardation.

Tic-Tac-Touch - The workstation

The finger painting metaphor used as the initial exercise for this project is basically a task where the user chooses a colour from a palette on one screen and finger-paints on an empty piece of white paper, in this case an empty white screen. The Tic-Tac-Touch system is not a computer application but a comfortable workstation. It is not only adapted to the users interaction and intellectual needs, but also for his physical needs or impairments. An example of this last point is the concern for accessibility for people in wheelchairs. The system will be described based on the use of the finger painting exercise.

The Tic-Tac-Touch workstation when using the finger painting application is divided in 4 major components:

1-The colour palette

This touchscreen monitor (15"), in the finger painting exercise, simulates a colour palette from which the user may choose the colour he or she wants to paint with. There are six different colours available on the palette screen: red, green, blue, a dark brown, a light yellow and a florescent colour. The presence, saturation, brightness and position of these colours are subjected to changes throughout the research activities. These changes strongly depend on the patterns of behaviour identified in the user. The computer application has a semi-intelligent reading ability to test the user if certain tendencies are identified. Some of the tendencies analysed are, for example, persistence in using a certain colour or no use at all of a colour. A more extensive explanation of the semi-intelligent character of the program is given further on in this paper. The colour palette is positioned on the right side of the user, however if needed, the monitor position may be changed to fulfil the ergonomic needs of a left-handed user.

2-The canvas

The canvas is essentially another touch screen but bigger than the colour palette one (19"). It works as the users blank piece of paper where he will finger paint. The digital canvas represents as close as possible the drawing that would result from a real life finger painting session with the same colours. Except for one detail. The colours will not be mixed as in real life. The digital canvas will be positioned on the left of the user but, yet again, if needed, it may be relocated on the right side of the user.

3-The table

The table upon which the two touchscreens are located is ergonomically designed according to some identified needs in possible users. Some of those ergonomic adjustments are:

Table top special design to enable comfortable access for people in wheelchairs; Table top cushioning to allow a comfortable position for who decides to lean on the table; Physical barrier supports for users with sudden non-controlled arm movements. 4 - The switch-lock monitor locator (not represented in the figure) The two touchscreens are positioned in a switch-lock monitor locator mechanism in the table at an adjustable 45º angle. This mechanism will allow the manipulation of the monitor location (left or right side of the user) and its position.

The non-present observer

The gathering of information for statistical analysis is possibly one of the most complicated tasks when doing research with people with mental retardation. The traditional method of compiling data for after exercise analysis would be what you might call, the three P method. Pencil, paper and patience. The researcher would observe the user and take down notes on the users actions. If needed he would time the users reactions and responses. In well-detailed scenarios the researcher would even have a data sheet with multiple choices ready to be ticked. And last, but not least, the researcher would have to have a great deal of patience to input all this data into a computer for analysis purposes and information backup. With the correct use of new technologies in the research field we are able to overcome a great deal of these set backs. If we pass on to the computer the responsibility of "taking down notes" we will be able to free extra time for other tasks and even increase the number of users involved in the study. The user will be confronted with the non-existence of an observer taking down notes, which would in many cases inhibit his behaviour. This will allow him do interact naturally with the system without having the impression that he is being watched or tested. Further more the internal real time and second precise annotation of the users actions will increase the number and precision of interaction related data saving it in an already digital format re-usable in work sheet programs such as Excel.

Information gathered

All of the users actions are registered internally while he interacts with the system, but what sort of information are we talking about? In this specific case, the finger painting exercise, there are two main groups of data referring to the user and his interaction patterns. The first group of information is essentially related to the user identification and the exercise session reference. Generically this group includes:



The second group of data that will be gathered in the finger painting exercise includes the following information and represents the appropriate metrics to evaluate usability issues correlated to user behaviour:

Testing and adapting

The use of software must be a dynamic relationship where the user adapts his or her behaviour to the applications mainframe interaction pattern. If this dynamic relationship becomes bilateral, and the software adapts itself to the users interaction patterns, the analysis of these adaptations will allow us to identify some of the users preferences and habits. The finger painting exercise does not adapt itself to the user but instead reads his interactions and if a certain action occurs, and if it exists in a predicted list of situations, then the application will take a pre-defined action in response. The telematic services must achieve levels of plasticity dependent of the flexibility of the individual using it. Only with this bilateral relationship will it be possible to optimise the adaptation of the service to the user behaviour patterns [Almeida et al99]. Some of the pre-defined actions are:

The relocation of the colour in the colour touchscreen (the application tries to test if the influence factor in the choice of the colour is its location); The manipulation of the colours brightness and saturation levels (the application tries to test if the influence factor in the choice of the colour depends on its brightness or saturation); The exchange of a colour, which is rarely used or not used at all, by another colour (the application tries to test if a new colour will interest the user more than the former one). For instance, if the user has applied the colour red in more than 50 percent of each drawing for the last 10 painting sessions, then the program will analyse possible responses for this type of behaviour – the dominance of the same colour in more than 10 sessions. A possible response may be the exchanging of the colour red by another colour, its relocation in the colour palette touchscreen or even the manipulation of its brightness and saturation. After probably 20 or 30 painting sessions the information gathered will give us an idea of which colours should be set aside as individual visual stimuli for future testing.

The target users

The analysis of the data gathered from the various painting sessions is also used as a correlation factor between the several target user groups involved in the research effort. Although users with severe mental retardation form the main target group, they may be divided in two groups that differ in the following aspects:


The correlation study includes a number of users between 20 and 50. The crossing of the data from their finger painting exercises will possibly result in the identification of a mainstream behaviour in this target group regarding tendencies and preferences in screen projected colours.

Final considerations

The Tic-Tac-Touch system and the finger painting exercise have already passed the idealisation phase and are now stepping into the implementation phase. As a very important tool for research in my PhD studies, the whole system is expected to be up and running at the end of this year. Afterwards it will be used in sessions in various schools and institutions with potential users that may, by there profile, be included in our target group. Results from these sessions and the outcome of the analysis of the data gathered from them are expected to supply us with ideas on which path to take on the use of colours in multimedia applications for people with severe mental retardation. The colours, which are identified as being generally preferred by the target group, will afterwards be used in simple multimedia applications. These programs will serve as beta testers for optimised colour palettes identified in the finger painting exercise.

Assuming the primacy of the user, understanding him and giving him the opportunity to use these new technologies should be our corner stone philosophy which will dictate the success, or not, of this study and the accuracy of its results [Redmond-Pile95]. We are living an era where the user is no longer labelled as the final owner of a product or service. The user must instead be understood as an active voice in the processes of creation and implementation of those same products and services. He must be the first to indicate the weaknesses and the strengths in tools created for his own use. Only through the true adoption of this dynamic development attitude will we be able to give the expression - information society for all – a true and global meaning.

References

[Sternberg & Spear85] Sternberg, R.J.&Spear, L.C. (1985) A thriarchic theory of mental retardation. In: "International Review of research in Mental Retardation", vol. 13, New York: Academic Press, pp 301-326.

[Erickson96]:Erickson, Thomas D. (1996) Working with Interface Metaphores. In "The art of human-computer interface design", edited by Brenda Laurel: Addison Wesley

[Almeida et al99]. Almeida, Margarida ; Raposo, Rui ; Silva, Lídia Oliveira ; Antunes, Maria João ; Almeida, Pedro ; Abreu, Jorge T. Ferraz de (1999) Understanding the user, his role and participation in the development of telematic service: transversal research efforts in the Department of Communication and Arts. In ICPPIT 99-International Conference on Public Participation and Information Technologies, Lisbon

[Redmond-Pile95] Redmond-Pile, David; Moore, Alan, (1995), Graphical User Interface Design and Evaluation, London: Prentice Hall;

Go to previous article 
Go to next article 
Return to 2000 Table of Contents 
Return to Table of Proceedings

Reprinted with author(s) permission. Author(s) retain copyright.