Go to previous article
Go to next article
Return to 1995 VR Table of Contents
D.J.Brown, D.S.Stewart and J.R.Wilson
Paper presented at the Virtual Reality and Persons with Disabilities
Conference, San Francisco, August 1995.
This paper will discuss the history of the Learning In Virtual Environments (LIVE) project, from conception to current new research strains. A chronological project development is presented, looking at the development of virtual environments (VE's) to teach life and communicational skills. This research and development phase is tempered with an account of the experiences of the use of the LIVE program in the main beta-test site, the Shepherd School in Aspley, Nottingham. The moral and ethical implications of the use of the program will be developed, together with the integration of this new teaching methodology into the curriculum for students with severe learning difficulties.
The Virtual Reality Application Research Team (VIRART) is based within the Department of Manufacturing Engineering and Operations Management, and was established in June 1991. Developing a broad base of Manufacturing, Education and Training applications, the team has captured several major grants, including sponsorship by the Health and Safety Executive to qualify and quantify the potential risks of the extended use of headset-based virtual reality.
Ever since the formation of VIRART, a central research and development thrust of the team has been the production of VE's for students with severe learning difficulties. Given an introduction to the Shepherd School by a mutual work colleague, striking commonalities between the components of a VE and the 'props" used in teaching were noted. Where VE's offered a platform for free-play, and self-directed activity interacting with three-dimensional computer generated models, the Shepherd School used theatre, role-play, modelling, aromatherapy and music to stimulate learning. It was conjectured that VE's could also bypass the abstracted symbols based method of teaching and provide a tool for experiential and proactive teaching that could bring into any one classroom a whole world of experience.
It was thought from the start that this approach would be particularly useful for life skills that were difficult to practise on a daily basis (for example, shopping, where it is difficult to arrange staff cover, and goodwill of the participating shop), and experiences that due to the disability of some students, are unlikely ever to be encountered, (for example, skiing). The team felt, however, that virtual experience was not a replacement for real world experience; only a preparation for it, especially when the experience in question was difficult to obtain of a regularity to build and reinforce necessary lessons. We had to acknowledge, however, that for some profound and multiply disabled students virtual experience may represent their only taste of a particular experience.
The development policy adopted at the beginning of the project still holds good today: all applications are suggested by the staff and pupils at the school participating in the project and all embryonic environments undergo the testing and redevelopment/refinement stage within an educational setting.
This policy has led to the production of two categories of VE: Experiential to help teach and reinforce basic life skills and Communicational to develop the use of Makaton Symbols. Primary testing of these VE's began with an assessment of whether students with severe learning difficulties could recognize and articulate the meaning of virtual objects that they knew and recognised in the real world. The success of this study was followed by a Population Stereotype study to investigate which input devices were most appropriate for use by students with a range of motor skills difficulties, to enable navigation around and interaction within VE's. It would be of little point to use VE's in support of the claim that they provide the most intuitive human- computer interface developed thus far if our user group cannot operate the necessary input devices. This comparative study of commonly used and adapted devices allowed the team to recommend a range of robust and useable devices for use by students with motor skills disabilities.
The next stage of testing was carried out in conjunction with the Department of Learning Disabilities, also based within the University of Nottingham. This team headed by Standen and Cromby will also report on our findings at this Conference, but briefly the experiential environments were used to investigate whether experience gained in a VE could generalize to a real world ability, whilst the communication environments were used to investigate whether the use of VE's could promote self directed activity. At the same time, work was undertaken to embed the development and use of future VE's in contemporary educational theory, combining this with continuing research and development, aiming to identify the components of a VE that influence learning and its transferability to real life situations.
The Shepherd School, Nottingham, is a large day school for pupils and students with severe and profound learning difficulties, run by Nottinghamshire Local Education Authority. There are 160 students aged 3 - 19, the large majority living at home with their families. In the UK all children are entitled to state education. Not all special education is the inclusive model but the entitlement is for all children. The Shepherd School forms part of the provision for children with special needs.
The School, named after an Elizabethan gentleman, Anthony Shepherd, is nearly 25 years old. These young people did not come under the education system in the UK until 1971.
The school serves mainly an inner city catchment area and there are large pockets of poverty, giving an additional disadvantage. The school population is changing as there are an increasing number of young people presenting with profound and multiple learning difficulties. In addition to a profound learning difficulty, they may have additional physical disabilities, may be blind, deaf or suffer with epilepsy. Some of these youngsters would probably not have survived to school age of 3, even 5 years ago. Indeed some may still die early and therefore it is essential that they are provided with vital and enriched experiences whilst at school. We are also noting a gender change as it seems to be girls who are surviving in this particular group,
The school seeks to promote the education for all students and enjoys a high profile in the community, particularly in the field of the Arts. Students, for instance, perform at the Edinburgh and other festivals. They work with all our major drama companies, ballet and opera companies and orchestras. We work closely with universities in developing work, including our neighbours at Nottingham University. We encourage strong links with similar youngsters in Europe, who in many cases do not enjoy the same advantages of education.
We have high expectations for our students and strive to provide an exacting, stimulating and useful curriculum. We are anxious that students should be given the skills to empower themselves and to take an active control of their own lives. Many students will always need some degree of support but we look for planned dependency, where the dependency can be reduced as much as possible and power restored to the student.
There appears to exist a strong mapping function between the potential use of a VE and contemporary developmental theories of education.
In terms of development of these electronic spaces for the purposes of special education much qualitative and quantitative research remains to be undertaken to determine a coherent and workable theory for its use.
At this stage, we are as Newton said: "just playing with pebbles on the beach at the edge of a sea of knowledge". We can, however, elucidate the relationship between what a VE can offer as an education forum and such theories:
(i) Self Directed Activity: Students with severe learning difficulties experience relatively little control over their environment, and may expect to play a passive role for most of the time. Within virtual environments our students are encouraged to be proactive, deciding where to go next and how (walking, skiing, driving, in a wheelchair) interacting with the objects to investigate their function, to see what noises they make. Bruner (1968) and Vygotsky (1978) have emphasised the importance of self-directed activity in their theories. Held and Hein (1963), in their work on perception pointed out the importance of self- directed activity. A VE can offer a 'sheltered space' for students to self-initiate actions especially where they have little experience or confidence to do so in the real world.
(ii) Motivation: Stuart and Thomas (1993) concluded from their research into the potential use of VE's in education that where as the age of television has bred passive, disengaged students with short attention spans, the use of VE's may be able to captivate students and foster their active involvement in their own education.
(iii) The role of play: Many children are considered vulnerable by parents and carers, but students with disabilities are often over- protected, sometimes not gaining sufficient experience for development (Shakespeare 1973). VE's offer a close electronic model of the real world, but mistakes made in them are far less likely to incite ridicule or peer-pressure. Play can be a major factor in liberating students from constraints, as pointed out by Vygotsky (1978). Clements (1987), implicates play in the acquisition of metacognitive abilities and in the development of a sense of self, both important areas for students with development disabilities. Students can also play 'let's pretend' and take on the characteristics and powers of virtual characters. The strong identification that players can feel with artificial characters in a computer database has led Laurel (1991) to assert that this is an example of the human capacity for mimesis , to which Aristotle attributed the soul-changing power of drama.
(iv) Experiential Learning: Many learning systems, on the market today, are described in terms of abstracted symbol systems (English, Mathematics, etc). Many students with developmental disabilities experience difficulties in acquiring and using such systems of disembedded thought (Donaldson, 1978). VE's, however, have their own natural semantics (Bricken 1995), a virtual house appears and functions very much like a real house. Concept attainment can occur through practical activities, bypassing the need for disembedded thought.
(v) Shared and Public experience: For special education, Virart has used desktop virtual reality systems for reasons of useability, affordability and health and safety considerations. These platforms display the environment in a public way, enabling learning in conjunction with an instructor. Vygotsky (1978) noted the importance of the social nature of learning by way of his concept of the "zone of proximal development". This is defined as the distance between a child's "actual development level as determined by independent problem solving" and the higher level of "potential development as determined through problems solving under adult guidance or in conjunction with more capable users".
Historically, other commentaries have also been offered on the cause and treatment of people with disabilities;
"This should have been a noble creature, he Hath all the energy which would have made A goodly frame of glorious elements Had they been wisely mingled; as it is It is an awful chaos - light and darkness And mind and dust - and passions and pure thoughts, Mixed, and contending without end or order; All dormant or destructive; he will perish, And yet he must not; I will try once more; For such are worth redemption; and my duty Is to dare all things for a righteous end: I'll follow him - but cautiously, though surely." -Byron
This verse from Byron, our local Nottingham poet, whilst in some ways giving a positive message of direction and support, has still the elements of an exterior control. And indeed this didactic method is typical of much education. Little credit is given to what the student has to contribute or to share in opinion or experience. We must always be mindful that education should not only enlighten but empower people.
The possibilities which VR might have for our education excites our imaginations. The medium seems a very vital way in which youngsters can not only engage in but also control their environment. We see two immediate developments, one as providing virtual experiences which could then be generalised in real life experiences. For others, particularly the most severely disabled, it would provide experiences which would never be achieved in real life.
Writing at the time of Byron's death in 1826, Froebel reminds us in The Education of Man, that 'Every human being, even as a child, must be recognised, acknowledged, and fostered as a necessary and essential member of humanity'. It was Froebel who stressed the importance of play and imagination in a young person's life. Given psychology credibility by Piaget, we are greatly aware of the importance of play as a vital part of each person's cognitive, social and emotional development.
It has already been noted that the Shepherd School places great store by the Arts; drama and imaginative play are greatly encouraged. We know that as part of the cognitive process children develop thinking and learning by doing; by giving opportunities for simulated experiences, young people are given the opportunity to explore numerous scenarios, evaluate their effectiveness or suitability and reassess their actions. Role-play activities allow for a 'safe' environment in which mistakes can be made and learnt from. One can imagine oneself in a whole range of situations, where one can be in control, or even make a range of decisions.
At the school we have a solar-visualization room which re-enacts several real life situations for students who because of their medical conditions find it very difficult to experience the real world - for instance they cannot effectively control their body temperature so that even on a fine day, they cannot be outside for very long. The room can recreate a seaside, a woodland, a riverside, sunny days, foggy days; one can be under the sun or going through the air. One can be at the fairground or in a turkish bazaar. There are a multitude of visual and auditory effects. Soundbeam has also been much used.The use of VE's then lends itself very well to this approach to teaching.
One area which was identified was the use of VE's in the teaching of Makaton signs and symbols. The Makaton communication system is an adapted vocabulary for British Signed Language, promoted and developed by Margaret Walker and others since the nineteen seventies. It is a system much focused in the education of those with severe and profound learning difficulties in the UK, also experience communication problems regardless of any hearing loss. It should be noted that these systems were in fact used with these young people in America in the 1820's and England in the 1830's so the principle is a very old one. The symbols are probably more widely used in practice as the manual skill required for signing still defeat many youngsters.
The first computer programme for the symbols was devised by David Kirby of Walsall Education Authority in the early 1980's. It was noted in those early days for the matching and sequencing activities that the young people were 'highly motivated and gained confidence'. It is not surprising that young people who are now brought up on new technology should also be motivated by the much more sophisticated, challenging and interactive programmes of VR. From there they can seek to develop and experiment with its usage. To paraphrase Froebel, "My teachers are the students themselves with their irresistible claims and I follow them like a faithful trustful scholar. It is of no consequence that precisely these programmes and these activities shall be used which have been suggested by myself. They have been put forward merely by way of example, others may find some more challenging or constructive purposes."
Using the develop-test-refine model, a number of experiential and communicational applications have been developed and distributed throughout Britain and Europe. Each application was suggested by staff or pupils at a special school or rehabilitation centre and, subsequently, tested and refined in such an educational setting. Each of the environments is now described:
Virtual House: Using 3-dimensional models (to produce rooms, furniture), scanned images (to produce familiar patterns and textures such as wood, carpets, wallpaper), sounds (running water, opening doors, television, radio), interactivity (opening doors, functioning radio, television, telephone, cooker, kettle) and embedded sequences (to guide the student on an appropriate way to make a cup of coffee), a virtual house has been developed in which life skills can be developed.
Within the house, a whole series of skills can be practised in isolation or in conjunction with other skills. For example, we might pretend it is morning time and upon getting up, check the weather outside (scanned textures attached to the window to give the impression of a sunny or rainy day) and then to the wardrobe to discuss and choose appropriate clothes for that day. After this the student may decide to go down stairs and make a cup of coffee. This involves filling and boiling the kettle, putting the coffee in the cups, switching the kettle off after boiling, filling the cup with hot water and, finally, adding milk. If a mistake is made, for example, attempting to boil the kettle before filling it with water, warning messages are given: 'Danger, fill the kettle with water before boiling', which can be written in Makaton symbols and repeated verbally. In this way the student is gently guided back on a safe and correct path once more. The student will have reached this position by their own autonomous action, throwing up vital lessons as the embedded sequence is revealed in a safe and controllable environment. Such pitfalls can be encountered and learned how to be dealt with before rejoining the correct learning path once more.
Virtual Supermarket: The virtual supermarket offers a forum for students to practise and develop shopping and monetary skills. The environment contains a series of shelves stacked with a range of common household goods and food. Using the joystick, a trolley can be pushed around and goods selected from the shelves using the mouse. The 'goods' will then disappear from the shelf and appear in the trolley. In this way, a shopping trip can be made, shopping lists can be discussed which may include healthy eating plans.
Once all the goods have been collected, the trolley can be pushed towards any of the checkouts. Each item can be placed on the checkout using the mouse and in doing so a price is displayed for each, displaying the total cost when all the items have been loaded. At this point, a series of coins appears in a box in the right-hand side of the screen. The mouse is used to select the correct coinage to pay for these goods and help develop the necessary monetary skills needed for some degree of independent living.
Virtual City: The virtual city consists of a series of office buildings, houses, shops, a cinema, a church, town square all built on a rectangular grid. The inter connecting roads on this grid, together with an underground tube system provide the transport infrastructure around the city. Cars and buses travel in an autonomous fashion around the city to simulate traffic allowing the students to build a road awareness and traffic sense whilst colonising this environment.
Each student can choose the option of 4 modes of movement around the city; the first is walking, sticking to the pavement, learning when and where it is safe to cross the road. The second is a wheelchair viewpoint, again travelling on the pavement, using ramps to cross the road, pedestrian crossings and investigating wheelchair access around a modern-day city. Thirdly, the driving car viewpoint, wherein the student takes the controls of the car (via the joystick) and drives around the city encountering road works, one-way systems, traffic lights and pedestrian crossings. Consequences of crossing a red or green light can be discussed and interactively investigated as the student decides on their own route. Many students can see for the first time, just how difficult it is for a driver to stop if a pedestrian crosses the road at other points than the pedestrian crossing, notably from between parked and stationary cars. For the profound and multiply disabled, we have provided the fourth viewpoint, as a passenger in a car travelling around the city.
This public and shared environment provides an opportunity for exchange between pupil and teacher, to discuss means of access around a modern day city and how a person with disabilities might best use the facilities that a city has to offer.
It is also possible to go down into the underground at one end of the city, board the underground train and alight at the other station at the other end of the city. Any journey in real life can be multimodel (peripatetic, car, bus, underground) and these alternatives can be practised in the virtual city.
Virtual Skiing: This VE transports the student to virtual St Moritz and is purely a leisure experience, but one that many of the students that we work with may never experience in real life due to profound and multiple disabilities, combined with limiting economic factors. Using the joystick, the student accelerates down the piste, circumnavigating the control gates. There are three speeds to choose from; novice, intermediate and advanced to cope with the developing skill of each virtual skier. Passing underneath the 'Fin' gate the student is drawn onto the ski-lift and transported back up to the top of the piste once more.
Many students, especially those in wheelchairs, respond extremely positively to the 'sense' of movement generated by this experience. It also provides good training in the use of the main navigation device around virtual learning environments; the joystick, as the students become so embroiled with the rapid navigation of the ski-gates.
One of the primary suggestions made by the Shepherd School at the first round of brainstorming special educational applications for VE's was the Makaton Development programme, consisting of some 350 two-dimensional symbols arrayed in eight increasing level of complexity. From the outset it was acknowledged that it was not whether this new system would provide a means to access to more rapid learning of Makaton symbols, but that it provided a new and stimulating teaching medium, important both to teachers and students. This is especially true in the light of the fact that some students are so challenged they might only learn ten symbols in their entire school life.
Within each virtual learning environment, the student will encounter four new Makaton symbols placed in separate 'warehouses'. The screen is split into 3 sectors as the viewpoints switch between warehouses. Sector 1 contains the hand-signs - a mannequin who signs and speaks the word relating to each Makaton symbol when activated with the mouse or touchscreen. Sector 2 contains the Makaton symbol itself, whilst sector 3 contains a range of 3-D, functional and interactive models that represent and encapsulate the meaning of the Makaton symbol. For instance, the mannequin might sign and say telephone' for the Makaton symbol telephone, whilst the student can roam around encountering a range of public, private and mobile telephones from which they can make and receive calls.
A fifth and final warehouse will contain four object representing each of the four new symbols recently encountered, together with the mannequin and a randomly selected Makaton symbol. The student is challenged to make an association between this symbol and the object it represents. A correct association will activate the 'thumbs up' sequence and be accompanied by a 'bugle roll', whilst an incorrect association sees the mannequin perform the 'thumbs down' sequence, accompanied by a lonesome last stand on the bugle.
Currently, the team has built warehouses containing some 50 Makaton symbols, drawn from the first four levels of the Makaton vocabulary.
PUPIL'S EXPERIENCE: Anecdotal evidence abounds on the positive response of our student population to their use of the LIVE programme. We thought it best for the students themselves to address the Conference via video and document their own experiences.
There is sufficient evidence that Information Technology is making a unique and valuable contribution to the learning of pupils with special educational needs. It enriches their learning experiences and enhances their access to a broader curriculum.
As part of this development VR has much to offer. For it to develop, it will be dependent upon the commitment of individual teachers, who require:
*Professional and technical In-service Training.
*Sufficient equipment for classroom use.
*Good quality information about VR programmes.
*Technical assistance and maintenance.
For students to access VR into the curriculum there will need to be further ways of accessing the programmes, other than the traditional input devices of spaceball, joystick and mouse. These may include touch screen and sound beam technology. Hall (1992) has already conducted an initial study of the use of input devices in VE's by students with special needs, recommending appropriate equipment for students with a wide range of motor skills abilities. This study is now being followed up and extended by Crozier in Brown et al (1995), incorporating a comparison of these more specialised devices.
There are many applications in the area of language and communication; it can encourage responses; allow for situations of cause and effect, in personal and social education, ensuring concentration, strengthening motivation, developing adults to work with others, to plan and solve problems. it gives everyone opportunities in independent activity.
In 1990, Her Majesty's Inspectorate in carrying out a survey into IT and special educational needs in schools noted few examples of recording of the progress of students in their use of IT. What records there were, made little significant contribution to planning or progression. There is a real danger here for validating the acquisition of what can be very effective apparatus. If there is to be a justification for its use, its educational function and value must be validated.
Staff may feel unsure about the technology. Most staff in our school will still feel uncertain. The same HMI reported the real need for teacher INSET to reinforce skills and give confidence. If staff do not appreciate the potential, the students will not be given appropriate access nor will they value its use.
Precision teaching and discrete recording have always had a high place within the school but there must be a purpose to this. We have all witnessed the era of the tick sheet - are we recording skills or understanding. We value the support of the University, who are really addressing the needs of the student, and the researchers. One is mindful, however, of the comments of Madame Montessori of over 50 years ago, who was concerned that some research 'did nothing' beyond promoting new problems or expressing difficulties but gave no solution of the problems which it ought to accomplish.
Testing, so far, has taken a three stage approach, stage one investigated whether the navigation and interaction tools were appropriate and useable by students with a range of motor skills difficulties and made recommendations on the ones that functioned reliably, easily and robustly.
Stage two used the Supermarket environment to investigate whether skills learned in a virtual environment could be transferred and generalized to real world activity. This testing stage was designed and conducted in conjunction with the Department of Learning Disability at the University of Nottingham, using a virtually trained group of shoppers in comparison with those who received no such training. The methodology, experimental design and results of this testing phase are presented in great depth by our colleagues Standen and Cromby, but in summary it was found that after an extensive period of virtual training, students could shop at faster rates and more accurately than those who had received no such training.
Stage three investigated the claim that the use of virtual environments promotes self-directed activity. Using the Makaton environments 'directive-time' plots were produced to compare instances of directives issued by teachers and student's own initiatives whilst using these environments over a recorded time period. In summary, it was found that as teacher issued directives fell, student initiatives increased, becoming confident and comfortable with their environment.
Whilst work continues apace with our experiential and communication environments, other more recent projects have blossomed and are briefly reviewed below:
The experiential environments described previously are aimed at students commonly between the ages of 4 - 18 years old with congenital disabilities. After these widespread demonstrations, the team received many requests to develop versions of these environments, which were age- appropriate, the older people with acquired disabilities to help in rehabilitation. So far, VIRART has developed Senior City, Kitchen, High Street and Bowls and these are currently being tested in a rehabilitation centre in England.
Autistic children, like other children are attracted to the visual images and sound stimulation from television, cartoons, video recordings, radio and recorded music. Many parents of autistic children employ the use of audio-visual stimulation to entertain and pacify them. These mediums, however, are passive and do not allow the subject to interact with the audio-visual stimuli and positively influence the events. VIRART is currently building a virtual interactive audio-visual house which enables the user to influence the outcome of events.
A virtual tool is being developed by VIRART which will enable the Nottingham Social Services Housing Department to anticipate problems before adapting existing housing stock to suit the needs of disabled residents.
You will note that we have used desk top VR. Why not head set based VR? It has been our contention so far that we do not know enough about the effects of VR on the human thought process. It is one thing for us to make the decision for ourselves; it is another to impose this on others in our care. This may seem to be contrary to what we have been saying about empowerment but there are ethical issues. Our students all suffer some form of brain damage, over half the school suffers with epilepsy. We need to know a great deal more before we can expose our students.
We would wish to close with a poem by Walt Whitman, which as a very positive message which speaks for our young people.
"I celebrate myself and sing myself, And what I assume, you shall assume, For every atom belonging to me as good belongs to you. I have heard what the folkes were talking; the talk of the beginning and the end. But I do not talk of the beginning or the end."
Bricken, M. Virtual reality learning environments: potentials and challenges. Computer Graphics 25(3), 1991.
Brown, D.J, Crozier, Eynon, A. & Kerr, S.J. New research at VIRART. To be published in the British Journal of Special Education, July 1995.
Bruner, J.S. Processes of cognitive growth: infancy. USA, Clark University Press, 1968.
Clements, J. Severe Learning Disability and Psychological Handicap. Wiley: London, 1987.
Donaldson, M. Children's Minds. London, Fontana, 1978.
Froebel, F. The Education of Man, 1826.
Hall, J. Explorations of population expectations and stereotypes with relevance to design. Undergraduate thesis, 1993.
Held, R. & Hein, A. V. Movement-produced stimulation in the development of visually guided behaviour. J. Comp. Physiol. Psychology 56, p872-876, 1963.
Laurel, B. Computers as Theater. Menlo Park, California: Addison Wesley, 1991.
Montessori, M. The Discovery of the Child, 1948.
Shakespeare, R. The Psychology of Handicap. Methuen, London, 1975.
Stuart, R. & Thomas, J.C. The implications of education in cyberspace. Multimedia Review, 2(2), p17-27, 1991.
Vygotsky, L.S. Mind in Society: the development of the higher mental processes. Cambridge, Mass., Harvard University Press, 1978.
Go to previous article
Go to next article
Return to 1995 VR Table of Contents
Return to Table of Proceedings