Web Browser Accessibility using Open-Source Software

Web Browser Accessibility using Open-Source Software http:// amico.sourceforge.net Željko Obrenović Jacco van Ossenbruggen Semantic Media Interfaces CWI, Amsterdam [email_address] http:// www.cwi.nl/~obrenovi /

Introduction

Accessible Web Browser requires novel interaction modalities

Speech input and output for blind users

Gestures, head movement for spinal cord injured

Multilingual translation…

Combination of modalities

Web is more and more interactive

Web 2.0 – promises much more interactivity…

But it is hard to go beyond modalities supported by the browser (screen, keyboard, mouse)

Some Existing Solutions

Browser Extensions

Easy to build by most of developers

Problem of integration of complex code

Specially Designed Browsers

Less functional, not very popular

Label users as "disabled“

Adapted Open Source Browsers

Complexity of development and maintenance of code

Firefox 1.5.0.2: 10,970 files, 2,172,520 lines of code

Limited by technologies used by browser

Potential of Open Source and Free Software

Available OSS components, however, may support novel interaction modalities

Computer vision: OpenCV, HandVu…

Speech: FreeTTS, MaryTTS, Sphinx-4…

Language tools: WordNet, ConceptNet…

Translation services…

Can we better exploit OSS solutions, and reuse them with Web (2.0) technologies?

Example: Speech control of Google Maps:

All you do is put it together?

Many prog. languages

Many APIs

Semantic and temporal differences

High-level Web XML-based APIs

Low-level APIs of the device world

Fault tolerance

Flexible integration

Why it is so hard to use OSS?

Our Solution: AMICO:WEB http://amico.sourceforge.net/

Service-based loosely-coupled integration

Encapsulation OSS components as local or remote services

Solves the language issue, many OSS already provide service interfaces

Run OSS service as a process, functionality through open API

Two parts:

AMICO: A daptable M ulti- I nterface CO mmunicator

Web Integration Interfaces

Web Browser Web Integration Interfaces AMICO OSS Components

AMICO A daptable M ulti- I nterface CO mmunicator

Publish-subscribe communication service:

share data & exchange messages

run components as services

More APIs (TCP, UDP, XML-RPC, OSC, SOAP, SQL…)

Fault tolerance by loosely coupled integration

Temporal and data bridging

Low-level and high-level variables and transformations

D ecl arative XML abstraction , rapid prototyping

Flexible integration and reuse of components

Web Browser Web Integration Interfaces AMICO OSS Components

AMICO:WEB – Browser Integration

Browser extensions (Firefox)

Based on SIMILE Java Firefox extension

Use AMICO TCP and UDP interfaces

AJAX

Use AMICO HTTP Interaface (XMLHttpRequest)

Applets and scripts

TCP connection

Also in combination with extension such as Mozilla GreaseMonkey }

AMICO Example 1 / 2

Translation service

BabelFish service  Text-to-speech

Problems of combining (legacy) Web services , local components and browser components :

BabelFish Web Service

3 TTS engines: FreeTTS (English), Festival (Dutch), Mary (German, Tibetan)

Local database (MySQL, MS Access)

Browser toolbar as an interface

AMICO Example 2 / 2

Camera-based modalities – interaction with 3D (VRML) Web content

Problem of temporal mapping

Fast low-level face detectors (simple numbers, 0.1s)

High-level and slower Javascript VRML browser API

" Under the hood " WordNet database AMICO Core TCP UDP WordNet HTTP Application specific XML-RPC SQL Firefox toolbar extension User dictionary English speech recognizer English TTS engine Dutch TTS engine German TTS engine ConceptNet Yahoo BabelFish translation service Face/gesture/motion detection AJAX XMLHttpRequest Python Java Java C++ C Java MS Access MySQL MonetDB Web (HTML) Service Javascript Java Javascript

Conclusion