1995 VR Conference Proceedings

Go to previous article 
Go to next article 
Return to 1995 VR Table of Contents 


Virtual Presence And Autonomous Wheelchair Control: An Update

Stephen J. Smith
Center for Self-Organizing Intelligent Systems
Electrical Engineering Department, Utah State University
UMC 4120
Logan, UT 84322-4120
Voice/Message:(801) 797-2923, Fax: (801) 797-3054
EMAIL: SLNGP@CC.USU.EDU

Dr. Robert W. Gunderson
Center for Self-Organizing Intelligent Systems
Electrical Engineering Department, Utah State University
UMC 4120
Logan, UT 84322-4120
Voice/Message:(801) 797-2924, Fax: (801) 797-3054
EMAIL: SNOWVAX@CC.USU.EDU

Dr. Ben A. Abbott
Electrical Engineering Department, Utah State University
UMC 4120
Logan, UT 84322-4120

Monica Joshi
Electrical Engineering Department, Utah State University
UMC 4120
Logan, UT 84322-4120

Abstract

During the past year and one-half, Utah State University's (USU)Center for Self-Organizing Intelligent Systems (CSOIS) and the USU Center for Persons with Disabilities (CPD) have been applying virtual reality technology to wheelchairs. This paper presents developmental progress in two areas: virtual presence wheelchair control, and virtual reality training of autonomous wheelchair steering systems. Emphasis in both of these projects is on affordability. Virtual presence wheelchair control allows a remotely stationed operator to steer an occupied wheelchair.

Virtual presence control is achieved by providing the remote operator with environmental feedback from the wheelchair. For example, virtual sight is provided by mounting twin cameras on the wheelchair. The cameras track the head movements of the remote operator to provide a complete 360 stereo view of the wheelchair's surroundings. While there are many other possible uses, CSOIS/CPD is currently focusing on the use of the system for remote emergency take-over situations. Virtual reality training of an autonomous wheelchair steering system creates a virtual room, hallway, building interior, etc., through which the operator steers a virtual wheelchair to a desired location. A satisfactory path is then downloaded to the control computer on-board the wheelchair, which takes over in the real world.

While the concept is not particularly original, designing a working, affordable system is challenging. The paper will address these issues. Introduction The use of Virtual Reality (VR) technology to help disabled persons is not a new concept. To date, however, it's full usefulness has not even remotely been realized. VR has been applied successfully in training new wheelchair user's to drive safely and efficiently. Use of VR in CAD-type architectural design has also been explored.

One area that has not received as much attention, however, is the use of VR to improve the controllability and safety of actual wheelchair designs. This paper presents research in two applications of VR to wheelchair control: Virtual Presence wheelchair control and VR training of Autonomous Wheelchair controllers. The design of an autonomous wheelchair controller must involve careful consideration of the needs of wheelchair users. Most wheelchair users do not want or need a completely autonomous vehicle but would appreciate the safety, efficiency, and convenience that such a vehicle could provide. A system that would allow "window-shopping" instead of concentration on obstacle avoidance would be highly desirable. A system that could intervene if the driver lost control or came too close to a dangerous obstacle such as a curb or a stairway would also be beneficial. Such a vehicle would require an on-board obstacle detection and avoidance system and would navigate based on intelligent decision-making algorithms.

The wheelchair described above is currently being developed by the Center for Self-Organizing Intelligent Systems (CSOIS) and the Center for Persons with Disabilities (CPD) at Utah State University (USU). This involves building a "test-bed" vehicle, designing various control algorithms, and "training" the system. At this stage of development, CSOIS has developed a teleoperable wheelchair as a "test-bed" for training and evaluation of various control algorithms. The wheelchair is teleoperable for an operator can drive it by viewing stereo (3-D) images of the wheelchairs' surroundings and by manipulating a joystick to maneuver the chair as desired. Teleoperation is a precursor or subset of virtual presence control.

By providing additional sensory feedback (such as audio, touch, spatial orientation, and smell) full virtual presence can be achieved. Virtual presence control could provide caregivers with control of a patients wheelchair in emergency situations, it could be used by therapists to train autonomous wheelchair controllers for their patients, it could be used by wheelchair service centers for assessment of repair needs, and architects could use such a chair for modification of buildings to meet ADA standards. Virtual Presence Wheelchair Control In general, virtual presence control involves control of a distant object by placing oneself in the same space as the object through electronic means.

The degree of presence can vary. For instance, receiving coded data from the object and using the data to make control decisions is one form of virtual presence control. A more sophisticated example involves placing the operator "virtually" in the driver's seat using virtual reality technology. Virtual reality "immerses" the operator and the object in a new, virtual space. Here, the virtual operator can interact directly with the virtual object by sensing environmental data with virtual senses and then imparting control commands to virtual objects using virtual body parts. Though this example exaggerates the abilities of current technology, precursors of this type of control do exist. Current virtual presence control is most often achieved by giving the operator visual presence in an objects' space. The operator then manipulates the object using visual cues and some type of electronic interface to the objects' mechanism.

In virtual presence wheelchair control the operator receives stereo visual information from two cameras mounted on the wheelchair. The images from the cameras are displayed on a computer monitor. The left and right images are displayed alternately at a rate of about 15 Hz. Specially designed glasses worn by the operator ensure that the correct image is seen by the correct eye. That is, the right eye is presented with the right image only and the left eye is presented with the left image only. In this way, the operator receives a stereo image of the wheelchair's surroundings. Steering is performed with a joystick at the control computer. Joystick movements are converted to number data at the control computer. The steering data is then sent by radio to the computer on-board the wheelchair. The on-board computer converts the numerical data to voltages which control the drive motors on the wheelchair. VR Trained Autonomous Wheelchair Control Several techniques currently exist for autonomous control of wheeled vehicles. These include fuzzy-logic based controllers, Vector Quantization (VQ) pattern recognition controllers, and neural-network based controllers. All of these techniques have been proven effective in controlling slow moving vehicles through fairly predictable settings. Wheelchair control involves movement through predictable settings and, while optimal speed is desired, slow movement through wheelchair "unfriendly" settings is certainly required, even by human controllers.

Virtual Reality training combined with an appropriate autonomous control scheme can give the user optimal speed in simple and predictable settings while providing assistance in difficult settings. Virtual reality training of an autonomous wheelchair steering system involves creating a virtual room, hallway, building interior, etc., through which the "trainer", a human operator, steers a virtual wheelchair to a desired location. A satisfactory path is then downloaded to the control computer on-board the wheelchair. The on-board computer uses the trained path data in mathematical calculations to recreate the motions of the virtual wheelchair on a real wheelchair in the real world. At this point, the virtual world consists of extruded blueprints of USU buildings.

Using AutoCAD(TM), the trainer draws the desired wheelchair path on a 2D blueprint. Then the path can be verified by "walking through" the extruded 3D version. The authors have found this to be a very effective and simple method for training the wheelchair. While training in the 3D environment is the ultimate goal, it may be a more difficult option. The reason for this seeming incongruency lies in the fact that wheelchairs are inherently difficult to drive, the sole reason for papers such as this one. By modeling the wheelchair in a virtual environment, we recreate this difficulty for the trainer. On the other hand, driving in 2D, from a "God's eye view", is very simple, straightforward, and fast.

There are several ways to use the trained path data to achieve control of the wheelchair. One of the simplest methods is called dead reckoning. Dead reckoning is a means of calculating a future position using the current position and knowledge of the dynamics of the moving body. This method requires development of a mathematical model of both the training environment and the wheelchair. The accuracy of the system will depend on the accuracy of the model. That is, the more accurate the model as a representation of the real world, the more accurate will be the wheelchair response to the trained response.

As explained above, an accurate model of the wheelchair is required for dead reckoning control. Unfortunately, popular wheelchair designs using slip steering, where the rear wheels control direction and speed and the front wheels simply swivel, are not easily modeled. As a result, initial testing for proof of the concept has been carried out on a modified wheelchair that uses a much more easily modeled design called ackerman steering (like a rear wheel drive automobile). The wheelchair has a single left and a single right turn ratio and moves at a constant velocity. This way, mathematical modeling is simple and accurate. The Virtual World The writers can think of several uses for this technology. For instance, a virtual presence controlled wheelchair could be used by family and caregivers of disabled persons in emergency situations. First-time users could be trained to drive in a more efficient and safe manner with a virtual presence equipped wheelchair.

Using virtual presence, repair services and wheelchair companies could perform long range diagnostics in a rapid, cost effective manner. Virtual presence wheelchairs could be used by architects to design accessibility modifications to existing buildings. Further, disabled persons could use virtual presence to operate other robotic assistive devices. These uses are illustrated in the following fictitious scenarios.

Consider a nursing home setting. Ida no longer has use of her legs due to an auto accident. She also experiences occasional epileptic seizures. Ida enjoys daily trips around town in her electric wheelchair. The wheelchair has been equipped for virtual presence control. Should Ida sense an oncoming seizure, she can press a button that alerts the caregivers at the nursing home that she needs help. Pressing this button also activates the virtual presence control system. A pair of cameras mounted on a motorized boom rise from a storage compartment and position themselves above Ida's head. Stereo images appear on a monitor in the nurses' station. Control of the wheelchair passes to a joystick in front of the monitor. The nurse can drive Ida out of dangerous situations or even drive her back to the nursing home using virtual presence.

Similarly, virtual presence could be used by nurses to monitor, locate, and retrieve patients with Alzheimer's disease or other types of dementia. Tony is a 6-year-old boy with Cerebral Palsy. Tony's Individual Education Plan (IEP) team has concluded that Tony needs an electric wheelchair for independence at school. They have begun to search for funding which could take more than one year. In the meantime they suggest that Tony begin learning to drive an electric wheelchair using virtual presence. Each day, Tony will spend one-half hour in front of a monitor using virtual presence to drive a wheelchair at the state's central training center. Further, the training center can use blueprints of Tony's home to create a virtual representation of the home. Tony can safely train to drive in his virtual home even before he has an actual wheelchair. When Tony receives funding and buys a chair, his physical therapist will help him fine-tune his driving skills.

Jenny is Quadriplegic. The only muscles she can control are in her face. She has been using a sip-and-puff technique to drive her electric wheelchair for at least ten years. Although Jenny has become quite efficient with sip-and-puff, the dented doorways and broken furniture in her home and the many near accidents outside the home attest to the shortcomings of the sip-and-puff method. Jenny has agreed to try a new self-navigating wheelchair system designed by the Center for Self- Organizing Intelligent Systems. This system is capable of learning its way around Jenny's home and responding to her commands to go to certain destinations in the home. Jenny's therapist will train the system in VR using a virtual representation of the home.

The same visual data that the therapist receives to drive the chair will be used as training data for the system. Physically disabled individuals who are blind could also benefit from this type of system.

Jordan, age 35, is an architect living in New York City. He has been involved in remodeling design for 10 years. Since the passage of the American's with Disabilities Act (ADA), Jordan has been called upon to remodel many of the city's older office buildings so that businesses can comply with the ADA. This includes redesign of entry and exit ways, bathrooms, some hallways, and most office cubicle areas. Five years ago, when Jordan was busy with his fist ADA compliance projects, the general approach to redesign involved reviews of ADA guidelines, meetings with consultation groups that included disabled persons, and guesswork. Unfortunately, in many cases, the redesign was found to be unsatisfactory after the work had already been completed. Jordan searched for a way to get a personal feel" for what it meant to be disabled in a world designed for non-disabled persons. When Utah State University approached Jordan with their virtual presence controlled wheelchair he thought it would be worth a try. Jordan discovered just what he needed. With the virtual presence controlled wheelchair, he could actually sit in a wheelchair, try to drive into a bathroom stall and drive out again, reach for doorknobs and elevator buttons with mechanical arms, and maneuver around an office cubicle. Furthermore, Jordan could do this all from his office workstation, making design modifications along the way. Jordan is still using the virtual presence wheelchair with very successful results. Recently, he has received a new Computer Aided Design (CAD) package which "builds" a virtual version of the building using the camera data received as the wheelchair is driven around the building. Jordan can make changes to the virtual building and then test those changes by driving a virtual wheelchair within the virtual building.

Michelle lives in a small town in southern Utah. She has been using electric wheelchairs for at least fifteen years. In fifteen years Michelle has owned three wheelchairs. They simply do not last more than five years because regular maintenance is nearly impossible. To get repair work done, Michelle must send the chair to a repair center in Salt Lake City, 300 miles away. Then she must wait, sometimes weeks, before her chair is examined. During this time, Michelle is at the mercy of her disability. She must hire attendants if she can (it is difficult to find a good attendant for only a few weeks). In short, Michelle is cut off from the world during these times. For these reasons, it is easier and less costly to run a wheelchair until it falls apart than to schedule regular maintenance. Virtual presence wheelchair control could help avoid many of these problems. For instance, wheelchair service centers could perform periodic inspections of wheelchairs in rural locations using virtual presence. Service technicians could drive the chair using external cameras (cameras mounted in a room with the wheelchair) and observe the wheelchair's behavior. They could receive diagnostic information from sensors placed strategically around the wheelchair, in its control system, and in its mechanisms. With this information, problems could be detected early, parts ordered, and an appointment scheduled for repair. All this could be achieved with a minimum amount of inconvenience to Michelle. The Real World The realization of the preceding fictitious scenarios would involve bringing together several existing technologies so that a useful, cost effective, and reliable device could be produced. Adequate technology exits to capture images, to manipulate mechanical limbs, and to control drive and steering links. However, when one attempts to do these things at a distance, the speed of effective transmission becomes the limiting factor. Transmission of image data, by far the most difficult barrier to be overcome in this application, has received increased attention in recent years. With the push towards a universal information protocol, the so-called "Information Super-Highway", the ability to send a large amount of information over wire at increased speeds has been a hot topic of research.

For example, just 3 years ago, AT&T introduced their first video-phone. This product boasted a frame-rate of 7 frames per second. USU has been developing data-compression technology for video-phone systems that provides frame-rates of up to 15 frames per second. One must keep in mind that these frame-rates are being achieved on dedicated phone-lines, not on the crowded internet. Capturing one frame of image data on the "net" can take from minutes to one hour, depending on the quality of the image, its size, and the method used to compress the data. The future is not dim but bright, however. Improvements in data compression and transmission are occurring at an increasing rate. Indeed, these improvements will be crucial if the powerful ideas promised by current technology (Virtual Reality, Information Super- Highway) are to become reality for the general populace. Modern assistive devices for the disabled, often requiring the most recent technological developments, will become possible as these ideas become reality.

Conclusion

The goal of this paper is not to present the results of an exhaustive research effort. However, it is intended to stimulate thought about how two current technologies, virtual presence control and virtual reality, could be useful to disabled individuals when contemporary roadblocks in engineering design become milestones of future development. A working virtual presence controlled wheelchair has been described and its possible usefulness has been illustrated using several fictitious scenarios.

Finally, the possibilities of these illustrations becoming realities have been explored in the context of current research and technological breakthroughs.

Powell, G.; McJunkin, T.; Gunderson, R.: "Autonomous Wheelchair Control by an Approximate Virtual Reality System", Proceedings of the Second Annual International Conference on Virtual Reality and Persons with Disabilities 1994, pp. 123-128.

Warner, D.; Sale, J.: "Interventional Informatics" , Interactive Technology and the New Paradigm for Healthcare, 1995, pp. 399-405.

Swan, J.;Stredney, D.; Carlson, W.; Blostein, B: "The Determination of Wheelchair User Proficiency and Environmental Accessibility Through Virtual Simulation", Proceedings of the Second Annual International Conference on Virtual Reality and Persons with Disabilities 1994, pp. 156-161.

Inman, D.; Peaks, J.; Loge, K.; Chen, V: "Teaching Orthopedically Impaired Children To Drive Motorized Wheelchairs in Virtual Reality", Proceedings of the Second Annual International Conference on Virtual Reality and Persons with Disabilities 1994, pp. 55-58.

Pin, F.; Watanabe, Y.:"Steps Toward Sensor-Based Vehicle Navigation In Outdoor Environments Using A Fuzzy Behaviorist Approach", Journal of Intelligent and Fuzzy Systems, vol. 1, 1993, pp. 95-107.

Go to previous article 
Go to next article 
Return to 1995 VR Table of Contents 
Return to Table of Proceedings 


Reprinted with author(s) permission. Author(s) retain copyright.