8 Open-Source Concept Coded Graphic Symbol support in OpenOffice.org


Published on

During the last 10 to 15 years the use of graphic symbols to support literacy development and access to text content has become increasingly widespread in special needs education and AAC (Augmentative and Alternative Communication) practices. This popularity is founded on a growing body of positive experience and research studies. It is also accompanied by the availability and use of a widening range of educational software tools (such as the Widgit_Communicate:-series, Clicker, BoardMaker-Speaking Dynamically Pro, and EdWord). But why should these methods and resources remain in the confined domain of special needs education? As part of the ÆGIS project, graphic symbol support for access to text is developed for the standard and open source office suite OpenOffice.org (OO.org). This task is a part of the ÆGIS ambition to include people with cognitive difficulties in the efforts towards more general accessibility in standard ICT environments. The graphic symbol support will be developed as a plug-in extension primarily for Writer in OO.org, and will build on the Concept Coding Framework (CCF) I suggested open standard for multimodal language support defined in the WWAAC project, and further developed within the SYMBERED and ÆGIS projects. When the user enters text – by ordinary letter-by-letter typing or by selecting and entering whole words (provided by Assistive Technology tools) – or loads a file, contained words will be matched against a concept database. Graphic symbol representations will be offered according to the user’s preferences, ranging from inline parallel text + symbol representation, using the Ruby Annotation format, to a word lookup service. The graphic symbol support will be integrated with the improved Text-to- Speech (TTS)I support within OO.orgI that will also be addressed in one of the ÆGIS tasks. Functionality will be evaluated and refined in three rounds of user pilot resting within ÆGIS.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

8 Open-Source Concept Coded Graphic Symbol support in OpenOffice.org

  1. 1. Mats Lundälv DART, Sahlgrenska Univ. Hospital Göteborg Sweden Open-Source Concept Coded Graphic Symbol Support in OpenOffice.org
  2. 2. Accessibility Everywhere – also in the cognitive domain !? <ul><li>The ÆGIS project: Accessibility Everywhere – Groundworks, Infrastructures, Standards </li></ul><ul><li>Addresses accessibility on the general level </li></ul><ul><li>Committed to include cognitive accessibility issues in the scope of its ambitions </li></ul><ul><li>However, this is the most complex and least addressed and established area of general accessibility efforts </li></ul><ul><li>Typically supported by special solutions and resources </li></ul><ul><li>Huge benefits if some more general infrastructures for cognitive support could be developed and widely provided </li></ul><ul><li>Focus on support for access to communication and language content – multimodal language support . </li></ul><ul><li>(Motivation, problem area) </li></ul>
  3. 3. <ul><li>Investigate ways to apply established methodologies in special solutions for cognitive support in the language area – in AAC and special needs education – on a more general level by: </li></ul><ul><li>Further develop and refine the Concept Coding Framework (CCF) technology for multi-modal and multi-lingual language and symbol support, defined in the WWAAC and SYMBERED projects (primarily for web platforms) for application on desktop and mobile platforms </li></ul><ul><li>Implement and test CCF based general symbol support with users and experts – in combination with other open a11y services (TTS etc) - for the open source and cross-platform desktop environment as a plugin extension component for OpenOffice.org </li></ul><ul><li>... and as an open source Java application for AAC support on a range of mobile platforms supporting Java (not covered here) </li></ul>Research Objectives
  4. 4. Research approach, Methodology <ul><li>Based on state-of-the art in graphic symbol support and defined use cases and personas; Research, develop and test what may be achieved with a generalistic approach with a CCF plugin extension for OO.org. Issues: </li></ul><ul><ul><li>Database format: complementing MySQL with JavaDB/Derby </li></ul></ul><ul><ul><li>Populating the database – for initially two freely available symbol systems; Bliss and ARASAAC – and two languages; English and Swedish. Refining methods for effective semi-automatic CCF ontology database population </li></ul></ul><ul><ul><li>Refinement of word/phrase-to-concept/symbol lookup – from quality of data and frequency priorities, to the addition of active language parsing and processing (per language) </li></ul></ul>
  5. 5. Research approach, Methodology <ul><ul><li>Scaling problems, response times, needs for optimisation, stripping of redundant data </li></ul></ul><ul><ul><li>Using RDF support in ODF 1.2 for concept code annotation </li></ul></ul><ul><ul><li>Resolving the use, or limitations in use, of the only partially supported Ruby Annotation format in OO.org </li></ul></ul><ul><ul><li>UI and interaction issues; What alternative modes of graphic symbol support could and might effectively and realistically be provided – iterative design and testing </li></ul></ul>
  6. 6. Major Outcomes / Results <ul><li>Current outcomes: </li></ul><ul><ul><li>The first alpha version of the CCF symbol support plugin for OpenOffice.org released for Pilot testing and feedback </li></ul></ul><ul><ul><ul><li>Only partial implementation of planned UI and interaction models </li></ul></ul></ul><ul><ul><ul><li>Database format: Complementing MySQL with JavaDB/Derby – completed (possibly other formats to be considered) </li></ul></ul></ul><ul><ul><ul><li>Early stage of RDF support in ODF 1.2 for concept code annotation </li></ul></ul></ul><ul><ul><ul><li>A first preliminary database population accomplished – Bliss and ARASAAC symbols – and English and Swedish. Needs lots of error checking, improvements and manual adjustments </li></ul></ul></ul><ul><ul><ul><li>Refining methods for effective semi-automatic CCF ontology database population – ongoing </li></ul></ul></ul>
  7. 7. Major Outcomes / Results <ul><li>Envisaged outcomes: </li></ul><ul><ul><li>The full version of the CCF symbol support plugin for OpenOffice.org released as a free OS exemplar component </li></ul></ul><ul><ul><ul><li>Full implementation of tested and revised UI and interaction models – including Ruby Annotation formatting and alternative levels and styles of symbol support </li></ul></ul></ul><ul><ul><ul><li>Support for more symbol systems – e.g. Mulberry symbols – and more languages; Spanish, Dutch ... </li></ul></ul></ul><ul><ul><ul><li>Refinement of word/phrase-to-concept/symbol lookup – as far as resources allow: Addition of active language parsing and processing (per language) </li></ul></ul></ul><ul><ul><ul><ul><li>Contacts for further co-operation about multilingual grammar based support for text to symbol translation have been established with the newly started 7 th FW project MOLTO www.molto-project.eu </li></ul></ul></ul></ul>
  8. 8. User Interface & Interaction design 1 <ul><li>UI design scetch </li></ul><ul><li>Including support for Ruby Annotation layout and handling of symbol representation ambiguity </li></ul>
  9. 9. <ul><li>Current actual UI design </li></ul><ul><li>No support for Ruby Annotation layout or handling of symbol representation ambiguity (apart from show all available alternatives option). </li></ul>User Interface & Interaction design 2
  10. 10. <ul><li>Control </li></ul><ul><li>Current design for turning the graphic symbol support (and concept coding) on and off: </li></ul>User Interface & Interaction design 3
  11. 11. <ul><li>Settings </li></ul><ul><li>Current design with support for Bliss and/or ARASAAC in optional priority, and for English and Swedish (partially) </li></ul>User Interface & Interaction design 4
  12. 12. CCF – the basic components: The Concept Coding Framework -1
  13. 13. CCF – an alternative view: The Concept Coding Framework -2
  14. 14. CCF – in operation: The Concept Coding Framework -3
  15. 15. Conclusions and Outlook <ul><li>At this fairly early stage of the development it is possible to conclude: </li></ul><ul><li>It will be possible to provide good basic OS multimodal language support in the form of a free plugin for concept coded graphic symbol support within OpenOffice Writer, possibly also within Impress and other applications of OO.org </li></ul><ul><li>We will be able to use the RDF support in the ODF 1.2 standard for the concept coding meta-data storage </li></ul><ul><li>The levels of refinement that may be achieved in disambiguation of language modality transformations, UI and interaction support remain to be further investigated, prototyped and tested </li></ul><ul><li>The current state-of-the-art software special software for the target user groups will not be made redunent, but an important complement will be provided opening for wider application in standard open source as well as proprietary environments </li></ul>