jonathanhassell@yahoo.co.uk
Accessibility innovation through gestural and sign-language interfaces
Prof Jonathan Hassell (...
1Accessibility
and innovation
jonathanhassell@yahoo.co.uk
Without these innovators
(and more)
the lives of disabled people
would be much impoverished
Jim Thatcher for the first
Screen Reader
Thanks to iheni.com
Ray Kurzweil for the first commercial
speech recognition (& much more)
Bill Kastner, WGBH and the BBC
for the first closed captioning
Gregg Vanderheiden for starting the
accessibility guidelines ball rolling
Tim Berners Lee for “the power of the web is
in its universality…”
(inspired by Mike Paciello)
The legislators behind
Section 508 (USA) and
Disability Discrimination Act Pt III (UK)
Rob Sinclair for making ATs
easier to create with MSAA and UIA
Maguire & SOCOG for giving the world
its first web accessibility test case
The WebAIM crew for getting the word out
and stoking the discussion
Julie Howell and the PAS/78 authors
for making accessibility more strategic
Shawn Lawton Henry for building a bridge
between accessibility & usability
Mick Curran & James Teh for making
screenreaders affordable with NVDA
Steve Jobs for
making
accessibility
built in as
standard in
Apple products
Jennison Asuncion for GAAD
jonathanhassell@yahoo.co.uk
And all those who
funded their work,
used what they created,
shared the news…
jonathanhassell@yahoo.co.uk
And …
jonathanhassell@yahoo.co.uk
As mobile technology enables
more people to be innovators
Why shouldn’t
your face be here…?
None of those guys were available… so you’ve got me: Jonathan
Hassell
• >13 years experience in accessibility and inclusio...
2What is innovation?
“Fresh thinking that creates value”
http://www.economist.com/specialreports/PrinterFriendly.cfm?story_id=9928154
“The process of translating an idea or
invention into a good or service that
creates value or for which customers
will pay...
Great innovations
come from…
User needs
Technology
possibilities
Sustainable
market/fundi
ng
Great
innovations
3“Where do you get your
ideas from…?”
jonathanhassell@yahoo.co.uk
How to source ideas…
From following
market trends
(e.g. everything
goes social…)
From listenin...
jonathanhassell@yahoo.co.uk
How inclusion helps ideation…
From following
market trends
(e.g. everything
goes social…)
From...
jonathanhassell@yahoo.co.uk
The problem of fixation…
jonathanhassell@yahoo.co.uk
How people try to free themselves from it…
‘Draw an alien’ to free yourself from conventional ...
jonathanhassell@yahoo.co.uk
Why not find a challenge closer to home?
Think about how a disabled person would use your prod...
jonathanhassell@yahoo.co.uk
The beauty of constraints…
© 2011 Centre for Business Innovation Ltd
- 34 -
Centre for Business Innovation
“Connected Communities, helping you Do mor...
jonathanhassell@yahoo.co.uk
How focusing on users takes you beyond inclusion…
From following
market trends
(e.g. everythin...
jonathanhassell@yahoo.co.uk
Audience research – focus groups, ethnographic studies etc.
Keep listening through iterative user-testing
Do initial
audience
research
User test to
get better
audience
research
Devel...
Use a user-centred development process to help you
– BS 8878
jonathanhassell@yahoo.co.uk
4Innovation through
Natural User Interfaces
Example 1: uKinect Sign Recognition
Speech recognition is cool
Speech translation is cooler
The ideal
It takes time… – cf. Siri
Speech recognition was able to
understand digits in the 1950s…
Siri’s intelligence has been
work...
The problem
We had
6 months…
User-focused approach
User needs
Technology
possibilities
Sustainable
funding/ma
rket
Great
innovations
Our breakthrough
technology enabler 
The way ahead
The way ahead
Looked for
people who
could benefit
from each of
our (baby)
steps…
Why choose Makaton-users
as our target audience?
British Sign Language Makaton
Thousands of signs (> 21,000) Hundreds of s...
User-Centred Approach
Engagement
through a
character and
humour…
Inspiration:
Talking Tom
Proof of concept solution
Watch the video at: http://vimeo.com/32022176
Phase Two
Users and contexts of use
Signing
e-Learning game
Users with comms difficulties
through LDs, Autism, stroke
Education Empl...
Product launching May-14
Hints of a new opportunity
“Boris was so engaging
that blind students were
also asking to use it to
learn to sign…”
Example 2: The Nepalese Necklace
A Movement Game for Blind and VI Children
The Nepalese Necklace Original concept
• The idea:
• using audio-games & Microsoft Kinect’s
gesture recognition to encoura...
The Nepalese Necklace UCD Approach
• Initial user-research
• found experts in the learning, and
representatives of the lea...
The Nepalese Necklace Validation
• Did final research to prove value
• in all contexts of use
(in homes as well as as scho...
Partner of the Year
Gamelab UK
If you
remember
one thing…
Listening to your diverse audiences needs…
identifies challenges…
but innovation often follows a challenge
So why shouldn’t
this be you?
Training &
support for
BS8878
Standards
Innovation
www.hassellinclusion.com
Strategy &
research
More Hassell Inclusion at CSUN-14:
Web Accessibility Myths for the mobile generation
• Do disabled people really
use ATs?
...
More Hassell Inclusion at CSUN-14:
7 Signs of maturing in accessibility and inclusion
• The accessibility industry is
agin...
Get in touch…
e: jonathan@hassellinclusion.com
t: @jonhassell
w: www.hassellinclusion.com
Accessibility innovation through gestural and sign-language interfaces
Accessibility innovation through gestural and sign-language interfaces
Accessibility innovation through gestural and sign-language interfaces
Accessibility innovation through gestural and sign-language interfaces
Accessibility innovation through gestural and sign-language interfaces
Upcoming SlideShare
Loading in...5
×

Accessibility innovation through gestural and sign-language interfaces

2,393

Published on

CSUN 2014 talk by Professor Jonathan Hassell describing how Hassell Inclusion, Gamelab UK, and Reflex Arc are using Natural User Interface technologies like Microsoft Kinect to create a whole new generation of assistive technologies based around the movements, gestures and signs different groups of disabled people make.

Two projects are described:
Nepalese Necklace movement games for blind and partially-sighted children that encourage blind and partially-sighted children to engage more readily with their early mobility training through making the body- and spatial-awareness exercises they have to perform the controls for motivational 3D audio-games;

uKinect sign language eLearning games to help people who use sign language to more easily transition into employment by enabling them to learn workplace-specific sign vocabularies using instructive video and our innovative Kinect sign-language recognition system.

NB. All videos in my CSUN presentation had captions, but it's not currently possible to caption the embedded videos in this slideshare. If you need access to the captioned videos, email jonathan@hassellinclusion.com

Published in: Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,393
On Slideshare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
0
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide
  • Digital TV has brought many advances for disabled and elderly people in recent years… access services - subtitles, signing, and audio descriptionIt’s brought problems too…“what box do I need… and how expensive is it?”“how do I find the programmes with the access services I need?”especially if the electronic programme guide doesn’t include speech…
  • Digital TV has brought many advances for disabled and elderly people in recent years… access services - subtitles, signing, and audio descriptionIt’s brought problems too…“what box do I need… and how expensive is it?”“how do I find the programmes with the access services I need?”especially if the electronic programme guide doesn’t include speech…
  • Digital TV has brought many advances for disabled and elderly people in recent years… access services - subtitles, signing, and audio descriptionIt’s brought problems too…“what box do I need… and how expensive is it?”“how do I find the programmes with the access services I need?”especially if the electronic programme guide doesn’t include speech…
  • And we found…
  • Working with the students to work out what they’d like…
  • Here’s a summary of what we were able to achieve in our Phase One.
  • Here’s a summary of what we were able to achieve in our Phase One.
  • 32 minutes to here
  • 32 minutes to here
  • Transcript of "Accessibility innovation through gestural and sign-language interfaces"

    1. 1. jonathanhassell@yahoo.co.uk Accessibility innovation through gestural and sign-language interfaces Prof Jonathan Hassell (@jonhassell) Director, Hassell Inclusion Visiting Professor, London Metropolitan University CSUN, San Diego, USA 19th March 2013
    2. 2. 1Accessibility and innovation
    3. 3. jonathanhassell@yahoo.co.uk Without these innovators (and more) the lives of disabled people would be much impoverished
    4. 4. Jim Thatcher for the first Screen Reader Thanks to iheni.com
    5. 5. Ray Kurzweil for the first commercial speech recognition (& much more)
    6. 6. Bill Kastner, WGBH and the BBC for the first closed captioning
    7. 7. Gregg Vanderheiden for starting the accessibility guidelines ball rolling
    8. 8. Tim Berners Lee for “the power of the web is in its universality…” (inspired by Mike Paciello)
    9. 9. The legislators behind Section 508 (USA) and Disability Discrimination Act Pt III (UK)
    10. 10. Rob Sinclair for making ATs easier to create with MSAA and UIA
    11. 11. Maguire & SOCOG for giving the world its first web accessibility test case
    12. 12. The WebAIM crew for getting the word out and stoking the discussion
    13. 13. Julie Howell and the PAS/78 authors for making accessibility more strategic
    14. 14. Shawn Lawton Henry for building a bridge between accessibility & usability
    15. 15. Mick Curran & James Teh for making screenreaders affordable with NVDA
    16. 16. Steve Jobs for making accessibility built in as standard in Apple products
    17. 17. Jennison Asuncion for GAAD
    18. 18. jonathanhassell@yahoo.co.uk And all those who funded their work, used what they created, shared the news…
    19. 19. jonathanhassell@yahoo.co.uk And …
    20. 20. jonathanhassell@yahoo.co.uk As mobile technology enables more people to be innovators
    21. 21. Why shouldn’t your face be here…?
    22. 22. None of those guys were available… so you’ve got me: Jonathan Hassell • >13 years experience in accessibility and inclusion • lead author of BS 8878 British Accessibility Standards • former Head of Usability & Accessibility, BBC • led work to embed accessibility across BBC web, mobile and IPTV production teams • won BIMA 2008 & Access-IT@Home awards for the accessibility features of BBC iPlayer • Product Manager of innovative products: • won IMS Global Learning Impact Award 2010 for MyDisplay • won ‘Best Usability & Accessibility’ BIMA 2006 for My Web, My Way • 3 x Bafta-nominated for breakthrough rich-media eLearning projects for disabled children
    23. 23. 2What is innovation?
    24. 24. “Fresh thinking that creates value” http://www.economist.com/specialreports/PrinterFriendly.cfm?story_id=9928154
    25. 25. “The process of translating an idea or invention into a good or service that creates value or for which customers will pay.” www.businessdictionary.com/definition/innovation.html#ixzz2iko2wNHH
    26. 26. Great innovations come from… User needs Technology possibilities Sustainable market/fundi ng Great innovations
    27. 27. 3“Where do you get your ideas from…?”
    28. 28. jonathanhassell@yahoo.co.uk How to source ideas… From following market trends (e.g. everything goes social…) From listening to users’ unmet needs and finding ways to meet them From encouraging your team to come up with ideas From following technology possibilities (Freeview => Youview)
    29. 29. jonathanhassell@yahoo.co.uk How inclusion helps ideation… From following market trends (e.g. everything goes social…) From listening to users’ unmet needs and finding ways to meet them From encouraging your team to come up with ideas From following technology possibilities (Freeview => Youview)
    30. 30. jonathanhassell@yahoo.co.uk The problem of fixation…
    31. 31. jonathanhassell@yahoo.co.uk How people try to free themselves from it… ‘Draw an alien’ to free yourself from conventional thinking
    32. 32. jonathanhassell@yahoo.co.uk Why not find a challenge closer to home? Think about how a disabled person would use your product…
    33. 33. jonathanhassell@yahoo.co.uk The beauty of constraints…
    34. 34. © 2011 Centre for Business Innovation Ltd - 34 - Centre for Business Innovation “Connected Communities, helping you Do more with Less” Engineering Design Centre The potential of inclusive design: OXO Good Grips • Well-known pioneer of Inclusive Design in the USA • Sam Farber’s wife, a keen cook, suffered from arthritis “Why do ordinary kitchen tools hurt your hands?” • First 15 products launched in 1990 • Sales growth over 35% per year from 1991 to 2002 • The line has now grown to over 500 products • Over 100 design awards received Look where asking the right question can get you…
    35. 35. jonathanhassell@yahoo.co.uk How focusing on users takes you beyond inclusion… From following market trends (e.g. everything goes social…) From listening to users’ unmet needs and finding ways to meet them From encouraging your team to come up with ideas From following technology possibilities (Freeview => Youview)
    36. 36. jonathanhassell@yahoo.co.uk Audience research – focus groups, ethnographic studies etc.
    37. 37. Keep listening through iterative user-testing Do initial audience research User test to get better audience research Develop minimal, flexible next version If more improvement justified, cycle…
    38. 38. Use a user-centred development process to help you – BS 8878
    39. 39. How mainstream products have arisen from innovative ‘beyond inclusion’ solutions
    40. 40. jonathanhassell@yahoo.co.uk 4Innovation through Natural User Interfaces
    41. 41. Example 1: uKinect Sign Recognition
    42. 42. Speech recognition is cool
    43. 43. Speech translation is cooler
    44. 44. The Funding Vision • Learners with disabilities may lack independence due to an inability to communicate by speech or due to lack of motor control • If signs and gestures can be easily learned, recognized and converted to digital data, a whole new world of opportunity opens up. TechDis, BIS, TSB SBRI ‘Making Waves’ competition
    45. 45. The ideal
    46. 46. The “breakthrough” inspiration • http://www.fastcoexist.com/1680081/a- device-to-translate-sign-language-into- speech and http://www.uh.edu/news- events/Fulbright/2012/may/0529MyVoice.ph p
    47. 47. It takes time… – cf. Siri Speech recognition was able to understand digits in the 1950s… Siri’s intelligence has been worked on for at least 10 years
    48. 48. The problem We had 6 months…
    49. 49. User-focused approach User needs Technology possibilities Sustainable funding/ma rket Great innovations
    50. 50. Our breakthrough technology enabler  The way ahead
    51. 51. The way ahead Looked for people who could benefit from each of our (baby) steps…
    52. 52. Why choose Makaton-users as our target audience? British Sign Language Makaton Thousands of signs (> 21,000) Hundreds of signs Individual sign vocab > 5,000 Individual sign vocab < 200 Long sequences of signs 1 or 2 sign sequences Sign quality fairly uniform Sign quality very variable, plus personal (idiosyncratic) signs Many competing teams innovating in this field Very few competing teams innovating in this field
    53. 53. User-Centred Approach Engagement through a character and humour… Inspiration: Talking Tom
    54. 54. Proof of concept solution Watch the video at: http://vimeo.com/32022176
    55. 55. Phase Two
    56. 56. Users and contexts of use Signing e-Learning game Users with comms difficulties through LDs, Autism, stroke Education Employment Independent Living Supporters of these users: colleagues, teachers, carers, parents
    57. 57. Product launching May-14
    58. 58. Hints of a new opportunity “Boris was so engaging that blind students were also asking to use it to learn to sign…”
    59. 59. Example 2: The Nepalese Necklace A Movement Game for Blind and VI Children
    60. 60. The Nepalese Necklace Original concept • The idea: • using audio-games & Microsoft Kinect’s gesture recognition to encourage blind and partially-sighted children to engage more readily with their mobility training • The project: • an inexpensive, 3 month Proof of Concept to investigate the idea’s potential in a concrete, testable way
    61. 61. The Nepalese Necklace UCD Approach • Initial user-research • found experts in the learning, and representatives of the learners • created a way of giving both an initial idea of what we were talking about, to get their attention and buy-in • asked questions to “get into their world” • then created what they needed/wanted • Iterative user-testing • we did this every couple of weeks • there’s no substitute for it
    62. 62. The Nepalese Necklace Validation • Did final research to prove value • in all contexts of use (in homes as well as as schools) • over longer periods of testing, without expert presence (over at least a week, without you propping the PoC up) • observed and interviewed users & experts to understand how they behave and feel about the PoC • Got the results on video • nothing else quite proves your case
    63. 63. Partner of the Year Gamelab UK
    64. 64. If you remember one thing…
    65. 65. Listening to your diverse audiences needs… identifies challenges… but innovation often follows a challenge
    66. 66. So why shouldn’t this be you?
    67. 67. • the full guide on how to transform your organisation to achieve the consistent creation of web sites and apps that are usable and accessible to all your customers, at the most efficient cost • with practical case-studies from leading accessibility experts worldwide, including: • Jennison Asuncion (Canada), • Debra Ruh & Jeff Kline (USA), • Andrew Arch (Australia) • David Banes (Qatar) • Axel Leblois (UN) for information on the book, free access to video case- studies, and a chance of winning the book for free Click here for chance to win book There’s more on accessibility innovation in my book
    68. 68. Training & support for BS8878 Standards Innovation www.hassellinclusion.com Strategy & research
    69. 69. More Hassell Inclusion at CSUN-14: Web Accessibility Myths for the mobile generation • Do disabled people really use ATs? • Is the most important accessibility issue for images alt-text? • Does inclusive design really benefit everyone? • Are the most important people in accessibility developers? • challenging some of the accepted views we hold that may no longer be true…
    70. 70. More Hassell Inclusion at CSUN-14: 7 Signs of maturing in accessibility and inclusion • The accessibility industry is aging and growing… • But is it maturing…? • how do you measure maturity in accessibility - in organisations, and as an industry? • from Tim Cook’s latest shareholder comments to the IAAP – here are 7 signs that we’re maturing as we grow…
    71. 71. Get in touch… e: jonathan@hassellinclusion.com t: @jonhassell w: www.hassellinclusion.com

    ×