2006 Conference General Sessions

ADDING ACCESSIBLE SELF-ASSESSMENT EXERCISES TO WEB-BASE!) LECTURES

 

 

Presenter(s)

Michael Litzkow
University of WisconsinMadison Street

1500 Engineering Drive
Madison WI 53706
Day Phone: 608-262-6122
Email: miitzkow@facstaff.wisc.edu

Presenter #2
Alice Anderson
University of WisconsinMadison Street

1210 W Dayton Street
Madison WI 53706
Day Phone: 608—262—2129
Email: alice.anderson@doit.wisc.edu

Introduction
eTEACH is a freely available authoring tool, developed for creating streaming-video- multimedia lectures that students view in a web—browser. Our efforts in making eTEACH accessible have been reported on previously [1]. Using eTEACH to provide out—of-class lectures allows faculty to engage students in active learning and problem-solving activities in class. In this paper we report on a significant enhancement to eTEACH that further defines it as an accessible learning tool. Our addition, called “Assessment Generator”, is a self-assessment quiz feature that focuses the student on the content and indicates their level of confidence
with the material. By asking questions related to the presentation and providing answers to those questions as part of the presentation, the student is more actively engaged. Because eTEACH is frequently used to deliver courses for engineering students, the ability to include complex mathematical content in the quizzes is critical to our mission. Making these quizzes accessible to students with disabilities, has turned out to be extremely challenging, but also extremely rewarding. This paper reports on the challenges that we faced, the mechanisms we developed to overcome those challenges, and our ongoing work in evaluating the effectiveness of these mechanisms. While we believe that what we have developed will benefit all students, our focus here is on how we met the particular needs of students who are blind or use the screen Leader UAWS.

 

Background
eTEACH lectures consist of streaming video and audio usually depicting the instructor (a talking head), animated slides (usually generated with PowerPoint), an interactive table of contents, and links to other web-based assets. Students can view these lectures at a time and place of their own choosing. This frees up instructors to spend more time actively engaging students in problem solving activities during class. We have done extensive surveying of students taking courses delivered with eTEACB and have found student reaction to be
largely positive [2). However some students found it too easy to view the materials at a superficial level and missed important points during the lecture. In a live—lecture setting an instructor might combat this problem by posing questions and engaging students in solving example problems immediately in class. We felt that adding a “self-assessment” feature to eTEACB could serve much of the same purpose in our web- based lectures. We chose to develop a self—assessment tool because we were interested in helping students assess and improve their own level of knowledge. The tool we created does not store any answers, and students are encouraged to attempt the exercises as many times as needed to become comfortable with the materials and concepts being presented. We began our work by creating assessments using hand-coded BTML pages. These hand-coded pages used various pieces of JavaScript and Cascading Style Sheets that we developed for this purpose. The assessments were designed by the professor in charge of the course, but implemented by student workers who received training. This system was neither easy for authors, nor was it accessible to JAWS users. However, it did give us an
initial mechanism for testing our idea of including self-assessments in web—based lectures. After using the self—assessments in a real class for a semester, we found that student attitudes and outcomes had improved noticeably [3]. We then decided to attempt solving the two main problems with our initial mechanism:
1. Providing an easy and intuitive way for instructors to create on—line quizzes.
2. Ensuring that the resalting material is accessible.

Making the Tool Easy to Use
We wanted to develop a graphical tool with a “point and click” interface which would be simple and intuitive for instructors to use. The tool would automatically do the translation to HTML in a way that would be accessible to JAWS users. Had we been willing to restrict our questions and answers to simple text, we could have easily created a web or desktop based form for this purpose. However, the inclusion of complex mathematical material would have been difficult using such a mechanism. The two most frequently used tools for creating documents with mathematical content are LaTeX and Microsoft Word with the MathType plug-in. Both mechanisms can be used to generate MathML, which can then be read by a screen-reader or other MathML aware software. Instructors in our college were more familiar with using Microsoft Word and MathType. Thus, we decided to develop a tool based on that technology. We developed the user interface of our tool as a Word template. When instructors load the template they get new
menu items and a new toolbar added to the familiar Word menus and toolbars to add or delete questions and answer choices for those questions. We expect that most questions will be multiple choice, but true and false and short answer questions are also allowed. Instructors choose the correct answer for multiple choice and true and false questions by selecting a radio button, and supply an answer for open-ended short answer questions. Students view the instructor’s answer after formulating their own. Instructors may supply hints” for some or
all questions and “feedback” for some or all answer choices. The “feedback” gives the instructor an opportunity to explain to the student why a particular answer is not correct. Output from the tool consists of HTML pages, and associated script and style sheet files. The quizzes pop up at pre—defined times during the eTEACH lecture. Students see feedback on whether their answer choices are correct immediately upon selecting an answer.

Solving the Accessibility Problems
Even without including the mathematical content, generating a quiz that is both accessible and useable by screen reader users is challenging. We concentrated our efforts on ensuring that our content would be useable by students using the JAWS screen—reader, and encountered several difficult problems:
* Preventing JAWS from returning to the top of the page or losing it’s place in the
document after giving feedback.
* Helping screen reader users avoid losing track of which answers go with which questions.
* Allowing students to move between questions easily.
* Avoiding specific browser requirements.
We created a “tips” page that explains the format to JAWS users and gives them hints on
how to navigate the quiz efficiently with JAWS. While our HTML formatting is very useable for JAWS users, it is also appealing for sighted users.

Current Work
Because we are not ourselves blind, we obtained continuing advice and feedback from users who are during the development of this project. However, those we worked with were not regular users of MathML and their ability to provide feedback about the usability of the math content in our example quizzes was limited. Also, since they had the benefit of direct help and coaching from us on how to best navigate the quiz forms, we feel it is important to determine whether these materials will be useable and comfortable for JAWS users who cannot help
directly. We feel that it is important to evaluate usability with a variety of users having differing levels of expertise in mathematics and differing levels of expertise with JAWS. For these reasons, we have begun a project where we will give sample self-assessment exercises to a larger number of JAWS users and obtain survey data from them regarding accessibility and usability. We have obtained permission from the human subjects review committee at our university to do this, and expect to be able to report on the results at publication

time.

Conclusions
In the short term, we have shown that complex interactive materials including sophisticated mathematics can be made accessible to JAWS users. Furthermore, we have created a tool which requires no extra effort or special knowledge on the part of the instructor who is creating those materials. We think that while the current generation of screen-readers is very good at reading static web content, it is lacking in ability to handle the kind of dynamic interactive content we want to deliver to our students. We have tried to work around these limitations using custom JAWS scripting for the eTEACH project as a whole, and by extremely careful HTML coding for this project. However, we find that the barriers presented to developers
who care about accessibility issues are extremely high. We think that this level of difficulty severely limits the amount of accessible educational content available to students, particularly in technical subjects like science, mathematics, and engineering. We will continue working on these issues in future projects.

References
“Making Multi-media Web—based Lectures Accessible - Experiences, Problems, and Solutions”; Michael Litzkow et. al, March 2004, 19’th Annual International Conference on Technology and Persons with Disabilities, 15—20 March 2004, Los Angeles CA.
2. “Reversing the Lecture/Homework Paradigm Using eTEACH Web-based Streaming Video
Software”; Julie Foertsch et. al., May 2002, Journal of Engineering Education.
3. “In-class Active Learning and Frequent Assessment Reform of Nuclear Reactor Theory Course”; Gregory Moses and Michael Litzkow, October 2005, [accepted for publication in the 35th ASEE/IEEE Frontiers in Education Conference, 19—22 October 2005, Indianapolis, IN]


Go to previous article
Go to next article
Return to 2006 Table of Contents


Reprinted with author(s) permission. Author(s) retain copyright