COD

  • CSUN Assistive Technology Conference Logo

    The Premier AT Conference

Task Aware Browsing: AI Assisted Web Access for Blind Users

Date & Time
Thursday, March 16, 2023 - 9:20 AM PST  
Location
Platinum 2  
Description
A completely new approach for blind users to access web pages is presented. It uses machine learning and artificial intelligence to overcome the problems encountered by these users with conventional browsers and screen readers, particularly on an unfamiliar web site. Finding the location of key features, when accessing Internet web sites, is difficult and time consuming. This approach provides a method for finding active controls, and tracking and managing the results from invoking these controls. The new approach is called TAB: Task Aware Browsing, For a given vertical domain, e.g. shopping, travel, banking, etc., a set of keyboard commands is created that allow the user to perform the essential functions in that domain. The browser then uses a model generated from machine learning to convert each command into the appropriate actions for a learned web site. It filters the results of page updates to present the screen reader with only the essential portion of rendered page that is responsive to the command, eliminating excess clutter. Work performed to date indicates that machine learning techniques will allow this approach to work on a significant number of web sites. User testing demonstrates that it provides an appreciable improvement in the speed with which blind users can accomplish desired tasks.  
Audience
  • Higher Education
  • Information & Communications Technology
  • Research & Development
Audience Level
Intermediate  
Session Summary (Abstract)
AI can provide blind users with a command line interface to perform tasks on many web sites. This solves the common problem these users experience in navigating the web. Experimental results demonstrate the feasibility of this approach.  
Primary Topic
Blind/Low Vision  
Secondary Topics
  • Artificial Intelligence (AI) & Machine Learning (ML)
  • Engineering
  • Web
 
Session Type
Journal Track  

Presenter

  • David Cane
    Perkins School for the Blind

Add to Outlook/iCal (How it works)

Back to Session List