The document discusses building accessible web browsers using open-source software. It proposes a solution called AMICO:WEB that uses a service-based architecture to loosely couple and integrate various open-source components like speech recognition and text-to-speech engines. This allows novel interaction modalities to be supported beyond just screen, keyboard and mouse. AMICO:WEB acts as an adapter between web browsers and open-source components running as local or remote services. Examples of using translation and camera-based modalities are provided.

1. Web Browser Accessibility using Open-Source Software http:// amico.sourceforge.net Željko Obrenović Jacco van Ossenbruggen Semantic Media Interfaces CWI, Amsterdam [email_address] http:// www.cwi.nl/~obrenovi /

2. Introduction Accessible Web Browser requires novel interaction modalities Speech input and output for blind users Gestures, head movement for spinal cord injured Multilingual translation… Combination of modalities Web is more and more interactive Web 2.0 – promises much more interactivity… But it is hard to go beyond modalities supported by the browser (screen, keyboard, mouse)

3. Some Existing Solutions Browser Extensions Easy to build by most of developers Problem of integration of complex code Specially Designed Browsers Less functional, not very popular Label users as "disabled“ Adapted Open Source Browsers Complexity of development and maintenance of code Firefox 1.5.0.2: 10,970 files, 2,172,520 lines of code Limited by technologies used by browser

4. Potential of Open Source and Free Software Available OSS components, however, may support novel interaction modalities Computer vision: OpenCV, HandVu… Speech: FreeTTS, MaryTTS, Sphinx-4… Language tools: WordNet, ConceptNet… Translation services… Can we better exploit OSS solutions, and reuse them with Web (2.0) technologies? Example: Speech control of Google Maps:

5. All you do is put it together? Many prog. languages Many APIs Semantic and temporal differences High-level Web XML-based APIs Low-level APIs of the device world Fault tolerance Flexible integration Why it is so hard to use OSS?

6. Our Solution: AMICO:WEB http://amico.sourceforge.net/ Service-based loosely-coupled integration Encapsulation OSS components as local or remote services Solves the language issue, many OSS already provide service interfaces Run OSS service as a process, functionality through open API Two parts: AMICO: A daptable M ulti- I nterface CO mmunicator Web Integration Interfaces Web Browser Web Integration Interfaces AMICO OSS Components

7. AMICO A daptable M ulti- I nterface CO mmunicator Publish-subscribe communication service: share data & exchange messages run components as services More APIs (TCP, UDP, XML-RPC, OSC, SOAP, SQL…) Fault tolerance by loosely coupled integration Temporal and data bridging Low-level and high-level variables and transformations D ecl arative XML abstraction , rapid prototyping Flexible integration and reuse of components Web Browser Web Integration Interfaces AMICO OSS Components

8. AMICO:WEB – Browser Integration Browser extensions (Firefox) Based on SIMILE Java Firefox extension Use AMICO TCP and UDP interfaces AJAX Use AMICO HTTP Interaface (XMLHttpRequest) Applets and scripts TCP connection Also in combination with extension such as Mozilla GreaseMonkey }

9. AMICO Example 1 / 2 Translation service BabelFish service  Text-to-speech Problems of combining (legacy) Web services , local components and browser components : BabelFish Web Service 3 TTS engines: FreeTTS (English), Festival (Dutch), Mary (German, Tibetan) Local database (MySQL, MS Access) Browser toolbar as an interface

10. AMICO Example 2 / 2 Camera-based modalities – interaction with 3D (VRML) Web content Problem of temporal mapping Fast low-level face detectors (simple numbers, 0.1s) High-level and slower Javascript VRML browser API

11. " Under the hood " WordNet database AMICO Core TCP UDP WordNet HTTP Application specific XML-RPC SQL Firefox toolbar extension User dictionary English speech recognizer English TTS engine Dutch TTS engine German TTS engine ConceptNet Yahoo BabelFish translation service Face/gesture/motion detection AJAX XMLHttpRequest Python Java Java C++ C Java MS Access MySQL MonetDB Web (HTML) Service Javascript Java Javascript

12. Conclusion AMICO:WEB – A Platform for Rapid prototyping of accessibility solutions in mainstream Web browsers Exploiting potential of OSS and free software Platform for future work and Web accessibility research Relation with W3C Rich Web Application Backplane Applications Improve accessibility of Web multimedia and e-learning systems Education: building accessibility solutions by undergraduate students Future work: Solving open issues: security, installation… Internet Explorer plugins , Google Web Toolkit… Proxy and Server Integration Mechanisms