COMPASS: SOFTWARE FOR
COMPUTER SKILLS ASSESSMENT
Presenter(s)
Heidi Koester
Koester Performance Research
2408
Antietam
Ann Arbor MI 48105
Day Phone: 734—663—4295
Fax: 734—663—8824
Email: hhk@umich.edu
INTRODUCTION
An accurate and efficient assessment of client abilities is a prerequisite for
a
successful computer access intervention. To support effective access
assessments, we have developed Compass, an easy—to—use software tool for
measuring a client’s skills needed for
various kinds of computer interaction, such as text entry, mouse or pointer
use, and information processing. Compass provides quantitative user performance
data that
complements the subjective observations made by clients and practitioners. It automatically
records speed and accuracy measures during task performance, leaving the
clinician free to observe more subjective aspects of client abilities, needs,
and preferences. Data are presented in an easy—to—understand format that can be
used in reports and letters of justification.
PROCRAM
DESCRIPTION
The current version of Compass includes eight skill tests in three input device
domains (printing device use, text entry, and switch use). Each test is
configurable to match it properly to the client’s needs. For example, the size
and color of text and objects can be adjusted if necessary. These
configurations are saved, allowing for efficient re—use in a subsequent session
if desired. During a test run, data regarding the speed and accuracy of user
actions are recorded. Following the test, Compass can generate a report,
summarizing results for the test and providing trial-by—trial detail if
desired. The currently available Compass tests are described in the list below.
1. Aim - Move mouse pointer into a target. Click (or dwell) to select target.
2. Drag — Move mouse pointer into a target. Click (or dwell) to select target.
Drag
target to a destination. Release target.
3. Menu - Select a specific item from a menu bar.
4. Letter — Enter a specific letter.
5. Word - Enter a specific word.
6. Sentence - Enter a specific sentence.
7. Switch - Press and release a switch.
S. Scan — Select a letter using a row-column scan matrix.
USABIL1TY
TESTING
Development of Compass has been guided by extensive user—centered research
activities. System requirements were defined and refined through a survey of
103 clinicians and in- depth discussions with 25 more. In four formal usability
trials, a total of 36 clinicians were able to perform a series of representative
Compass activities with no external help and without major errors. In a recent
beta test, the overall usability rating averaged 4.6 on a scale of 1 — 5 (5
being the best score), with a 95% confidence interval of [4.4, 4.7]. The
overall usability rating was based on responses to 10 survey questions
regarding ease of learning, efficiency of use, etc. Results from these tests
support the conclusion that Compass has a high degree of utility and usability.
CLINICAL
APPLICATION
Appropriate use of Compass helps a clinician to:
1. Diagnose difficulties with an existing interface; 2. Evaluate and compare
performance with candidate access systems; 3. Plan training interventions; 4.
Track changes in a client’s abilities over time; and 5. Measure the effectiveness
of an intervention. Using Compass can enhance the objectivity and accuracy of
access assessments as well as the efficiency and quality of associated reports.
Members
of the project team who provide computer access services have successfully
integrated Compass into their evaluations. Prior to meeting with a client, an
entire session can be “scripted” by configuring multiple skill tests. During
the session, skill tests can be easily duplicated or modified in response to
observations of the client’s performance and behavior. An unanticipated benefit
of Compass is that it allows the clinician to spend more time looking at the
client and less time looking at the computer screen, because Compass provides
detailed information about the client’s performance. This allows the clinician
to focus on issues such as how much effort the client is exerting, the strategy
the client is using, and the position the client is maintaining. After each
test, the clinician can review the results with the client to clarify tradeoffs
between different devices, configurations, and strategies. These results also
become an integral part of the final report provided to third-party payers.
NEXT
STEPS
Compass is currently being prepared for commercial release. We expect that it
will be commercially available by the time of this presentation in early 2006.
ACKNOWLEDGMENTS
This work is supported by NIH grant 2R42 NS3625202A1. We also thank the
clinicians who have responded to our surveys and participated in usability
testing.
REFERENCES
1. Ashlock G., Koester H., LoPresti E., McMillan N., and Simpson R.,
“User-centered
Design of Software for Assessing Computer Usage Skills,” 26th Annual Conference
on
Rehabilitation Engineering (RESNA), Atlanta, GA, June 2003.
2. Koester H., LoPresti, E. . “Compass: Software for Assessing Computer Usage
Skills”, SUN Conference on Technology for People with Disabilities,
3. LoPresti E., Koester H., and McMillan N. “Tools for Assessing Computer
Access Skills,” Proceedings of ASSETS 2002,
Go to previous article
Go to next article
Return to 2006 Table of Contents