Your SlideShare is downloading. ×
Deep Accessibility: Adapting Interfaces to Suit Our Senses
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Deep Accessibility: Adapting Interfaces to Suit Our Senses

427
views

Published on

Citation: …

Citation:

@article{Harper2012uq,
Abstract = {Disabled people typically use methods of `sensory translation' to access a Web-page via assistive technology. These technologies conventionally render screen content under the direction of the user into a form that can be perceived by that user -- in effect the interface and content are adapted to suit their sensory requirements -- but simple sensory translation is not enough.
Why is this -- and how can things be better? In this talk we touch on accessibility, sensory transcoding, multi-talker systems, auditory perception, and Neuroscience to help us in our search for equivalent interactive experiences tailored to the sensory modality of the user.},
Author = {Simon Harper},
Date-Added = {2013-02-15 10:31:27 +0000},
Date-Modified = {2013-02-15 10:39:41 +0000},
Howpublished = {Slideshare},
Journal = {Invited Talk - Technical Superior Insitute, LaSIGE, Lisbon, Portugal},
Month = {September},
Title = {Deep Accessibility: Adapting Interfaces to Suit Our Senses - http://goo.gl/VT5BE},
Url = {\url{http://www.slideshare.net/simon-harper/adapting-sensory-interfaces}},
Year = {2012},
doi={10.6084/m9.figshare.678330},
Bdsk-Url-2 = {http://dx.doi.org/10.6084/m9.figshare.678330},
Bdsk-Url-1 = {http://www.slideshare.net/simon-harper/adapting-sensory-interfaces}}

Published in: Technology, Business

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
427
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Sensory Translation Transcoding Auditory Perception Neurophysiology Next? Deep Accessibility Adapting Interfaces to Suit Our Senses Simon Harper University of Manchester 07 September, 2012 Deep Accessibility 1 / 31
  • 2. Sensory Translation Transcoding Auditory Perception Neurophysiology Next?The Story... Sensory Translation Transcoding Content Driven Transcoding Experience Driven Transcoding Modality Driven Transcoding Auditory Perception Cocktail Party Problem Properties of Sound Multi-Talker Display The Ear Neurophysiology Signal Transmission The Auditory Cortex http://goo.gl/VT5BE Neuro Plasticity Next? Deep Accessibility 2 / 31
  • 3. Sensory Translation Transcoding Auditory Perception Neurophysiology Next?Adapting Interfaces to Suit Our Senses Disabled people typically use methods of ‘sensory translation’ to access a Web-page via assistive technology. These technologies conventionally render screen content under the direction of the user into a form that can be perceived by that user – in effect the interface and content are adapted to suit their sensory requirements – but simple sensory translation is not enough. Why is this – and how can things be better? Deep Accessibility Sensory Translation 3 / 31
  • 4. Sensory Translation Transcoding Auditory Perception Neurophysiology Next?Transcoding “A category of technologies to transform inaccessible content to accessible content on the fly” To Accomplish Approaches Text Magnification; Syntactic: such as removing images; Colour Scheme Semantic: rearrangements and Changes; fragmentation; Serialisation; Annotation: created by a reader; and Text Insertion; Generated: annotations by CMS. Page Rearrangement; and Simplification. Deep Accessibility Transcoding 4 / 31
  • 5. Sensory Translation Transcoding Auditory Perception Neurophysiology Next?Content Driven Transcoding Screen-Scraping - 1990’s Originally content was ‘made accessible’ via a method called screen-scraping which used the visual rendering to create accessible content, by creating an off-screen model (a representation of the GUI). Screen-scraping was problematic because it was often wrong, did not take account of structure, and could not form accurate semantics between elements of the content. DOM Analysis - 2000’s Document Object Model analysis took over from screen-scraping and enables accurate structural semantics to be created as all elements and attribute values are available to the assitive technology. Deep Accessibility Transcoding 5 / 31
  • 6. Sensory Translation Transcoding Auditory Perception Neurophysiology Next?IBM Homepage Reader (HTML Annotation - 2000) Before After Deep Accessibility Transcoding 6 / 31
  • 7. Sensory Translation Transcoding Auditory Perception Neurophysiology Next?Structural-Semantics for Accessibility and DeviceIndependence (CSS Annotation - 2008) SADIe Transcoding Method Deep Accessibility Transcoding 7 / 31
  • 8. Sensory Translation Transcoding Auditory Perception Neurophysiology Next?Structural-Semantics for Accessibility and DeviceIndependence (CSS Annotation) . c n n C e i l n a v , d i v . cnnHeaderNav {uom− s t r u c t u r a l − r o l e : L i n k L i s t ; } input . cnnInput {uom− s t r u c t u r a l − r o l e : S e a r c h E n g i n e ; } d i v . CNNhomeBox , o l . c n n M o s t P o p u l a r , d i v#c n n T o p S t o r i e s . . . {uom− s t r u c t u r a l − r o l e : Chunk ; } d i v#c n n H e a d e r R i g h t C o l u l {uom− s t r u c t u r a l − r o l e : PageSummary ; } Deep Accessibility Transcoding 8 / 31
  • 9. Sensory Translation Transcoding Auditory Perception Neurophysiology Next?SaaS Transcoding (Code Analysis) - 2011 WIMWAT / SCWeb2 - Subtle Transcoding Deep Accessibility Transcoding 9 / 31
  • 10. Sensory Translation Transcoding Auditory Perception Neurophysiology Next?Readability - 2011 Before After Deep Accessibility Transcoding 10 / 31
  • 11. Sensory Translation Transcoding Auditory Perception Neurophysiology Next?Evernote Clearly - 2012 Before After Deep Accessibility Transcoding 11 / 31
  • 12. Sensory Translation Transcoding Auditory Perception Neurophysiology Next?Experience Driven Transcoding - 2010 ‘Content Driven Transcoding’ is focused on transforming the content based on its representation in the DOM; however ‘Experience Driven Transcoding’ goes a step further and attempts to transform the content based on both its representation, and the predicted experience of the user into an equivalent sensory experience . BBC News with AoI’s Deep Accessibility Transcoding 12 / 31
  • 13. Sensory Translation Transcoding Auditory Perception Neurophysiology Next?Example – Order Table: Areas of Interest (AoI) – BBC News (s = Seconds) ID Area of Interest Vis. Order Mean Fix. Time Linear Order 1 Header 1 0.7 s 1 2 BBC Ticker Static 3 8.3 s (Lost) 11 3 BBC One Boss 2 1.9 s 4 4 LHS Menu Top 6 11.9 s 2 5 Other Top Stories 4 9.1 s 5 6 LHS Menu Bottom 8 14.7 s 3 7 Features Views Analysis1 5 11.3 s 6 8 Video and Audio 9 15.3 s 7 9 Have Your Say 7 13.1 s 9 10 Around the World 10 17.7 s 8 11 Features Views Analysis2 11 28.2 s 10 Deep Accessibility Transcoding 13 / 31
  • 14. Sensory Translation Transcoding Auditory Perception Neurophysiology Next?Example – Graph Ranked Order Comparison Deep Accessibility Transcoding 14 / 31
  • 15. Sensory Translation Transcoding Auditory Perception Neurophysiology Next?SASWAT 2010 http://goo.gl/ty7OR Deep Accessibility Transcoding 15 / 31
  • 16. Sensory Translation Transcoding Auditory Perception Neurophysiology Next?IBM aiBrowser - 2010 IBMs aiBrowser Deep Accessibility Transcoding 16 / 31
  • 17. Sensory Translation Transcoding Auditory Perception Neurophysiology Next?VICRAM & EIVAA - 2011 Visual Complexity Rankings & Visual Aesthetics Rankings VICRAM Heat Map of Visual Complexity Deep Accessibility Transcoding 17 / 31
  • 18. Sensory Translation Transcoding Auditory Perception Neurophysiology Next?Modality Driven Transcoding ‘What the HECK?’ ‘Content Driven Transcoding’ is focused on transforming the content based on its representation in the DOM; ‘Experience Driven Transcoding’ attempts to transform the content based on both its representation, and the predicted equivalent experience of the user; ‘Modality Driven Transcoding’ adds another step, attempting to transform the content based on its representation, the predicted equivalent UX, and tailored to the sensory modality of the user; Deep Accessibility Transcoding 18 / 31
  • 19. Sensory Translation Transcoding Auditory Perception Neurophysiology Next?Auditory Perception – ‘Cocktail Party Problem’ Been talking about translating parallel visual experience into a serial auditory one. BUT auditory perception is parallel too. ‘The separation of two simultaneously spoken messages’, in which Cherry first poses the question ‘how do we recognise what one person is saying when others are speaking at the same time (the ‘cocktail party problem’)?’ is key. This can only be useful work in the domain of blindness, situation impairment, and accessibility in that it may be possible to convey limited Web page information much faster because of the ability to comprehend highly parallel speech. Cherry, E. (1953). Some Experiments on the Recognition of Speech, with One and with Two Ears The Journal of the Acoustical Society of America, 25 (5) DOI: 10.1121/1.1907229 Deep Accessibility Auditory Perception 19 / 31
  • 20. Sensory Translation Transcoding Auditory Perception Neurophysiology Next?Properties of Sound Deep Accessibility Auditory Perception 20 / 31
  • 21. Sensory Translation Transcoding Auditory Perception Neurophysiology Next?Multi-Talker Display – Brungart We now have between seven and nine people intelligibly multi-talking at the same time based on spatial location and voicing. This can only be useful work in the domain of blindness, situation impairment, and accessibility in that it may be possible to convey limited Web page information spatially, dynamically, and with a high degree of comprehension at seven (or nine) times faster because of the ability to comprehend highly parallel speech. Brungart, D., & Simpson, B. (2005). Optimizing the spatial configuration of a seven-talker speech display ACM Transactions on Applied Perception, 2 (4), 430-436 DOI: 10.1145/1101530.1101538A. W. Bronkhorst. The cocktail party phenomenon: A review of research on speech intelligibility in multiple-talker conditions. Acta Acustica united with Acustica, 86:117–128(12), January/February 2000. Deep Accessibility Auditory Perception 21 / 31
  • 22. Sensory Translation Transcoding Auditory Perception Neurophysiology Next? M. F. Bear, B. W. Connors, and M. A. Paradiso.The Ear Neuroscience: Exploring the Brain. Lippincott Williams & Wilkins, 2006. Deep Accessibility Auditory Perception 22 / 31
  • 23. Sensory Translation Transcoding Auditory Perception Neurophysiology Next?Frequency Detection M. F. Bear, B. W. Connors, and M. A. Paradiso. Neuroscience: Exploring the Brain. Lippincott Williams & Wilkins, 2006. Deep Accessibility Auditory Perception 23 / 31
  • 24. Sensory Translation Transcoding Auditory Perception Neurophysiology Next?Neurophysiology - Signal Transmission M. F. Bear, B. W. Connors, and M. A. Paradiso. Neuroscience: Exploring the Brain. Lippincott Williams & Wilkins, 2006. Deep Accessibility Neurophysiology 24 / 31
  • 25. Sensory Translation Transcoding Auditory Perception Neurophysiology Next?The Auditory Cortex M. F. Bear, B. W. Connors, and M. A. Paradiso. Neuroscience: Exploring the Brain. Lippincott Williams & Wilkins, 2006. Deep Accessibility Neurophysiology 25 / 31
  • 26. Sensory Translation Transcoding Auditory Perception Neurophysiology Next?Language - Positron Emission Tomography M. F. Bear, B. W. Connors, and M. A. Paradiso. Neuroscience: Exploring the Brain. Lippincott Williams & Wilkins, 2006. Deep Accessibility Neurophysiology 26 / 31
  • 27. Sensory Translation Transcoding Auditory Perception Neurophysiology Next?So What? Why Does this Matter Under used areas of the Brain develop differently; Repurposed for different effect; Occurs as Brain develops upto 18–21; called ‘Neuro Plasticity’. Deep Accessibility Neurophysiology 27 / 31
  • 28. Sensory Translation Transcoding Auditory Perception Neurophysiology Next?Neuro Plasticity H. Burton. Visual cortex activity in early and late blind people. Journal of Neuroscience, 23(10):4005-4011, May 2003. Deep Accessibility Neurophysiology 28 / 31
  • 29. Sensory Translation Transcoding Auditory Perception Neurophysiology Next?Modality Driven Transcoding Revisited More Work 1. Can frequency bounding to Auditory Cortex’s tonotopy enhance parallelisation –and therefore distinguishability– of both talkers and sonified interfaces? Deep Accessibility Next? 29 / 31
  • 30. Sensory Translation Transcoding Auditory Perception Neurophysiology Next?Modality Driven Transcoding Revisited More Work 1. Can frequency bounding to Auditory Cortex’s tonotopy enhance parallelisation –and therefore distinguishability– of both talkers and sonified interfaces? 2. Can congenitally blind users, and blind users who have became advantageously blind up to 18, perceive more talkers –in a multi-talker system– than sighted users? Deep Accessibility Next? 29 / 31
  • 31. Sensory Translation Transcoding Auditory Perception Neurophysiology Next?Modality Driven Transcoding Revisited More Work 1. Can frequency bounding to Auditory Cortex’s tonotopy enhance parallelisation –and therefore distinguishability– of both talkers and sonified interfaces? 2. Can congenitally blind users, and blind users who have became advantageously blind up to 18, perceive more talkers –in a multi-talker system– than sighted users? 3. Is any of this practically applicable in a Web engineering setting; will the User Agent support these modal transcodes and how? Deep Accessibility Next? 29 / 31
  • 32. Sensory Translation Transcoding Auditory Perception Neurophysiology Next?Deep Accessibility Not Just Not just about the technology; Not just about Vague UX; Not HCI as Psychology or Sociology Deep Accessibility Next? 30 / 31
  • 33. Sensory Translation Transcoding Auditory Perception Neurophysiology Next?Deep Accessibility Is Also About It’s about Accessibility as Science About Accessibility as Physiology Accessibility as Neuroscience Deep Accessibility Next? 30 / 31
  • 34. Sensory Translation Transcoding Auditory Perception Neurophysiology Next?Any Questions? Contact (vCard) Citations (BibTex) http://goo.gl/yzJFx http://goo.gl/ny2pD Deep Accessibility Next? 31 / 31