ADDING ACCESSIBLE
SELF-ASSESSMENT EXERCISES TO WEB-BASE!) LECTURES
Presenter(s)
1500
Engineering Drive
Madison WI 53706
Day Phone: 608-262-6122
Email: miitzkow@facstaff.wisc.edu
Presenter
#2
1210
W Dayton Street
Madison WI 53706
Day Phone: 608—262—2129
Email: alice.anderson@doit.wisc.edu
Introduction
eTEACH is a freely available authoring tool, developed for creating
streaming-video- multimedia lectures that students view in a web—browser. Our
efforts in making eTEACH accessible have been reported on previously [1]. Using
eTEACH to provide out—of-class lectures allows faculty to engage students in
active learning and problem-solving activities in class. In this paper we
report on a significant enhancement to eTEACH that further defines it as an
accessible learning tool. Our addition, called “Assessment Generator”, is a
self-assessment quiz feature that focuses the student on the content and
indicates their level of confidence
with the material. By asking questions related to the presentation and
providing answers to those questions as part of the presentation, the student
is more actively engaged. Because eTEACH is frequently used to deliver courses
for engineering students, the ability to include complex mathematical content
in the quizzes is critical to our mission. Making these quizzes accessible to
students with disabilities, has turned out to be extremely challenging, but
also extremely rewarding. This paper reports on the challenges that we faced,
the mechanisms we developed to overcome those challenges, and our ongoing work
in evaluating the effectiveness of these mechanisms. While we believe that what
we have developed will benefit all students, our focus here is on how we met
the particular needs of students who are blind or use the screen Leader UAWS.
Background
eTEACH lectures consist of streaming video and audio usually depicting the
instructor (a talking head), animated slides (usually generated with
PowerPoint), an interactive table of contents, and links to other web-based
assets. Students can view these lectures at a time and place of their own
choosing. This frees up instructors to spend more time actively engaging students
in problem solving activities during class. We have done extensive surveying of
students taking courses delivered with eTEACB and have found student reaction
to be
largely positive [2). However some students found it too easy to view the
materials at a superficial level and missed important points during the
lecture. In a live—lecture setting an instructor might combat this problem by
posing questions and engaging students in solving example problems immediately
in class. We felt that adding a “self-assessment” feature to eTEACB could serve
much of the same purpose in our web- based lectures. We chose to develop a
self—assessment tool because we were interested in helping students assess and
improve their own level of knowledge. The tool we created does not store any
answers, and students are encouraged to attempt the exercises as many times as
needed to become comfortable with the materials and concepts being presented.
We began our work by creating assessments using hand-coded BTML pages. These
hand-coded pages used various pieces of JavaScript and Cascading Style Sheets
that we developed for this purpose. The assessments were designed by the
professor in charge of the course, but implemented by student workers who
received training. This system was neither easy for authors, nor was it
accessible to JAWS users. However, it did give us an
initial mechanism for testing our idea of including self-assessments in
web—based lectures. After using the self—assessments in a real class for a
semester, we found that student attitudes and outcomes had improved noticeably
[3]. We then decided to attempt solving the two main problems with our initial
mechanism:
1. Providing an easy and intuitive way for instructors to create on—line
quizzes.
2. Ensuring that the resalting material is accessible.
Making
the Tool Easy to Use
We wanted to develop a graphical tool with a “point and click” interface which
would be simple and intuitive for instructors to use. The tool would
automatically do the translation to HTML in a way that would be accessible to
JAWS users. Had we been willing to restrict our questions and answers to simple
text, we could have easily created a web or desktop based form for this
purpose. However, the inclusion of complex mathematical material would have
been difficult using such a mechanism. The two most frequently used tools for
creating documents with mathematical content are LaTeX and Microsoft Word with
the MathType plug-in. Both mechanisms can be used to generate MathML, which can
then be read by a screen-reader or other MathML aware software. Instructors in
our college were more familiar with using Microsoft Word and MathType. Thus, we
decided to develop a tool based on that technology. We developed the user
interface of our tool as a Word template. When instructors load the template
they get new
menu items and a new toolbar added to the familiar Word menus and toolbars to
add or delete questions and answer choices for those questions. We expect that
most questions will be multiple choice, but true and false and short answer
questions are also allowed. Instructors choose the correct answer for multiple
choice and true and false questions by selecting a radio button, and supply an
answer for open-ended short answer questions. Students view the instructor’s
answer after formulating their own. Instructors may supply hints” for some or
all questions and “feedback” for some or all answer choices. The “feedback”
gives the instructor an opportunity to explain to the student why a particular
answer is not correct. Output from the tool consists of HTML pages, and
associated script and style sheet files. The quizzes pop up at pre—defined
times during the eTEACH lecture. Students see feedback on whether their answer
choices are correct immediately upon selecting an answer.
Solving
the Accessibility Problems
Even without including the mathematical content, generating a quiz that is both
accessible and useable by screen reader users is challenging. We concentrated
our efforts on ensuring that our content would be useable by students using the
JAWS screen—reader, and encountered several difficult problems:
* Preventing JAWS from returning to the top of the page or losing it’s place in
the
document after giving feedback.
* Helping screen reader users avoid losing track of which answers go with which
questions.
* Allowing students to move between questions easily.
* Avoiding specific browser requirements.
We created a “tips” page that explains the format to JAWS users and gives them
hints on
how to navigate the quiz efficiently with JAWS. While our HTML formatting is
very useable for JAWS users, it is also appealing for sighted users.
Current
Work
Because we are not ourselves blind, we obtained continuing advice and feedback
from users who are during the development of this project. However, those we
worked with were not regular users of MathML and their ability to provide
feedback about the usability of the math content in our example quizzes was
limited. Also, since they had the benefit of direct help and coaching from us
on how to best navigate the quiz forms, we feel it is important to determine
whether these materials will be useable and comfortable for JAWS users who
cannot help
directly. We feel that it is important to evaluate usability with a variety of
users having differing levels of expertise in mathematics and differing levels
of expertise with JAWS. For these reasons, we have begun a project where we
will give sample self-assessment exercises to a larger number of JAWS users and
obtain survey data from them regarding accessibility and usability. We have
obtained permission from the human subjects review committee at our university
to do this, and expect to be able to report on the results at publication
time.
Conclusions
In the short term, we have shown that complex interactive materials including
sophisticated mathematics can be made accessible to JAWS users. Furthermore, we
have created a tool which requires no extra effort or special knowledge on the
part of the instructor who is creating those materials. We think that while the
current generation of screen-readers is very good at reading static web
content, it is lacking in ability to handle the kind of dynamic interactive
content we want to deliver to our students. We have tried to work around these
limitations using custom JAWS scripting for the eTEACH project as a whole, and
by extremely careful HTML coding for this project. However, we find that the
barriers presented to developers
who care about accessibility issues are extremely high. We think that this
level of difficulty severely limits the amount of accessible educational
content available to students, particularly in technical subjects like science,
mathematics, and engineering. We will continue working on these issues in
future projects.
References
“Making Multi-media Web—based Lectures Accessible - Experiences, Problems, and
Solutions”; Michael Litzkow et. al, March 2004, 19’th
Annual International Conference on Technology and Persons with Disabilities,
15—20 March 2004, Los Angeles CA.
2. “Reversing the Lecture/Homework Paradigm Using eTEACH Web-based Streaming
Video
Software”; Julie Foertsch et. al., May 2002, Journal of Engineering Education.
3. “In-class Active Learning and Frequent Assessment Reform of Nuclear Reactor
Theory Course”; Gregory Moses and Michael Litzkow, October 2005, [accepted for
publication in the 35th ASEE/IEEE Frontiers in Education Conference, 19—22
October 2005, Indianapolis, IN]
Go to previous article
Go to next article
Return to 2006 Table of Contents