SlideShare a Scribd company logo
1 of 99
Accessible by
       Voice
   Or: how to make a tiny corner of the
internet not want to punch you in the face
Talking to the Dragon
                                                          •   Dragon Naturally
                                                              Speaking Home
                                                              £79.99

                                                          •   Dragon Naturally
                                                              Speaking Premium
                                                              £149.99

                                                          •   Dragon Naturally
                                                              Speaking Professional
                                                              £549.00

                                                          •   Dragon Dictate for
                                                              Mac £129.99




(I don’t make any money off Dragon. I wish I did ;-)

There are others, including dictation software built into Mac OS & Windows, and rudimentary
dictation / control software built into Android, iOS and Windows Phone 7. There are also
some highly specialised systems (eg: medical), and a few other dictation-only solutions as
well.

As far as I’m aware, Dragon is the only system that has reasonable voice control of desktop
computers as well as dictation capabilities.
Using Dragon:
Working with Text
Text input




So how does it work? You talk into a microphone, Dragon does some magic (involving maths),
and text appears in the target application. But that’s just transcription; what if you want to
correct what you dictated?
Text input




                                h
                         a h bla
                 Blah bl




So how does it work? You talk into a microphone, Dragon does some magic (involving maths),
and text appears in the target application. But that’s just transcription; what if you want to
correct what you dictated?
Text input



                                                                    Blah blah
                                                                    blah
                                h
                         a h bla
                 Blah bl




So how does it work? You talk into a microphone, Dragon does some magic (involving maths),
and text appears in the target application. But that’s just transcription; what if you want to
correct what you dictated?
Editing




Dragon keeps an internal representation of the text that has been dictated.
Editing




                             sh
                  Wh en I wi
                             ud
                  upo n a clo




Dragon keeps an internal representation of the text that has been dictated.
Editing



                                    When I wish
                             sh
                  Wh en I wi        upon a cloud
                             ud
                  upo n a clo




Dragon keeps an internal representation of the text that has been dictated.
Editing



                                                                    When I
                                    When I wish                     wish upon a
                             sh
                  Wh en I wi        upon a cloud
                  upo n a clo
                             ud                                     cloud




Dragon keeps an internal representation of the text that has been dictated.
Editing



                                                                     When I
                                    When I wish                      wish upon a
                  Delete
                         “wish”     upon a cloud                     cloud




It applies voice editing commands to its internal representation, and *then* issues editing
commands to the target application.
Editing



                                                                       When I
                                     When I dance                      dance upon
                    dance            upon a cloud                      a cloud




To do this, it keeps track of where the text cursor is in the target application.
Editing



                                                                      When I
                                   When I dance I                     dance I
                              t
                       ocumen      dance for me
                Cache d                                               dance for
                                                                      me




If things get out of sync (if you use the keyboard to edit, for instance) you can ask Dragon to
re-read what’s in the application. This is also useful when you load a new document.
Editing




Sometimes this goes wrong, particularly with apps that haven’t properly implemented
accessibility. Good apps on Mac: Textmate, Scrivener. Bad: Chrome. Somewhere in the
middle: Mail. Things that can go wrong range from random letters appearing, to overwriting
parts of words. Sometimes Dragon can’t even get back in sync with the application.
Editing




                              h
                Blah b lah bla




Sometimes this goes wrong, particularly with apps that haven’t properly implemented
accessibility. Good apps on Mac: Textmate, Scrivener. Bad: Chrome. Somewhere in the
middle: Mail. Things that can go wrong range from random letters appearing, to overwriting
parts of words. Sometimes Dragon can’t even get back in sync with the application.
Editing




                                 h
                                     Blah blah blah
                Blah b   lah bla




Sometimes this goes wrong, particularly with apps that haven’t properly implemented
accessibility. Good apps on Mac: Textmate, Scrivener. Bad: Chrome. Somewhere in the
middle: Mail. Things that can go wrong range from random letters appearing, to overwriting
parts of words. Sometimes Dragon can’t even get back in sync with the application.
Editing




                                  Blah blah blah                  Blah blah
                              h
                Blah b lah bla
                                                                  blahJ




Sometimes this goes wrong, particularly with apps that haven’t properly implemented
accessibility. Good apps on Mac: Textmate, Scrivener. Bad: Chrome. Somewhere in the
middle: Mail. Things that can go wrong range from random letters appearing, to overwriting
parts of words. Sometimes Dragon can’t even get back in sync with the application.
Automatic
                         Capitalisation




Dragon is designed (mostly) for dictating prose, so it understands that new sentences get
capital letters. If it doesn’t notice that you’ve moved from one input to the next, it may not
think you’re in a new sentence. (Conversely, sometimes it thinks you’re at the start of a new
sentence but you don’t want to be.) There are commands to force or suppress capitalisation
of the next word.
Automatic
                          Capitalisation




                              o
                  Iw anted t
                             u
                   m ak e y o
                     happy




Dragon is designed (mostly) for dictating prose, so it understands that new sentences get
capital letters. If it doesn’t notice that you’ve moved from one input to the next, it may not
think you’re in a new sentence. (Conversely, sometimes it thinks you’re at the start of a new
sentence but you don’t want to be.) There are commands to force or suppress capitalisation
of the next word.
Automatic
                          Capitalisation



                                    I wanted to
                              o
                  Iw anted t      make you happy
                             u
                   m ak e y o
                     happy




Dragon is designed (mostly) for dictating prose, so it understands that new sentences get
capital letters. If it doesn’t notice that you’ve moved from one input to the next, it may not
think you’re in a new sentence. (Conversely, sometimes it thinks you’re at the start of a new
sentence but you don’t want to be.) There are commands to force or suppress capitalisation
of the next word.
Automatic
                          Capitalisation



                                                                      Field 1:
                                    I wanted to                       I wanted to
                              o
                  Iw anted t      make you happy
                             u
                   m ak e y o                                         Field 2:
                     happy                                            make you happy




Dragon is designed (mostly) for dictating prose, so it understands that new sentences get
capital letters. If it doesn’t notice that you’ve moved from one input to the next, it may not
think you’re in a new sentence. (Conversely, sometimes it thinks you’re at the start of a new
sentence but you don’t want to be.) There are commands to force or suppress capitalisation
of the next word.
Express Editor




Dragon features an Express Editor which allows you to create a piece of text, editing as you
go, then insert it at the end. More tightly bound to Dragon, it makes fewer mistakes.
Express Editor




                              ah
                 blah blah bl




Dragon features an Express Editor which allows you to create a piece of text, editing as you
go, then insert it at the end. More tightly bound to Dragon, it makes fewer mistakes.
Express Editor




                                h
                                    blah blah blah
                 blah b lah bla




Dragon features an Express Editor which allows you to create a piece of text, editing as you
go, then insert it at the end. More tightly bound to Dragon, it makes fewer mistakes.
Express Editor




                                h
                                    blah blah blah
                 blah b lah bla


                                     Express Editor:
                                     blah blah blah


Dragon features an Express Editor which allows you to create a piece of text, editing as you
go, then insert it at the end. More tightly bound to Dragon, it makes fewer mistakes.
Express Editor




                                   blah blah blah




Once it’s transferred, if you need to do further editing you can cache it back into Dragon.
Express Editor




                                    t
                                        blah blah blah
                            the tex
                t ransfer




Once it’s transferred, if you need to do further editing you can cache it back into Dragon.
Express Editor



                                                                      Text field:
                                    t
                                        blah blah blah                blah blah
                            the tex
                t ransfer
                                                                      blah




Once it’s transferred, if you need to do further editing you can cache it back into Dragon.
Correcting Dictation



                                                                                                    Ye Highlands and ye
                                                                                                    Lowlands,
                                                                                                    Oh, where hae ye been?
                                                                                                    They hae slain the Earl
                                                                                                    O' Moray,
                                                                                                    And Lady Mondegreen.




     * Sylvia Wright (1954). “The Death of Lady Mondegreen”. Harper’s Magazine 209 (1254): 48-51.




Dragon also has features for correcting dictation errors…
Correcting Dictation



                                                                                                    Ye Highlands and ye
                                                                                                    Lowlands,
                                                                                                    Oh, where hae ye been?
                                     d
                              ct “An
                                                                                                    They hae slain the Earl
                        corre                                                                       O' Moray,
                            Lady                                                                    And Lady Mondegreen.
                                     n”
                         Mon  degree




     * Sylvia Wright (1954). “The Death of Lady Mondegreen”. Harper’s Magazine 209 (1254): 48-51.




Dragon also has features for correcting dictation errors…
Correcting Dictation



                                                                                                    Ye Highlands and ye
                                                     Correction options                             Lowlands,
                                                  1. And Lady Mondegreen                            Oh, where hae ye been?
                                     d
                              ct “An
                                                                                                    They hae slain the Earl
                                                  2. And laid him on the green
                        corre                                                                       O' Moray,
                            Lady                  3. and lady mondegreen                            And Lady Mondegreen.
                                     n”
                         Mon  degree




     * Sylvia Wright (1954). “The Death of Lady Mondegreen”. Harper’s Magazine 209 (1254): 48-51.




Dragon also has features for correcting dictation errors…
Spelling Things Out




…spelling out words that Dragon doesn’t know and so on. Text use can be faster than
keyboard if trained well and used with discipline.
Spelling Things Out



                                                                  Cinemato-
                                                                  graphy by




…spelling out words that Dragon doesn’t know and so on. Text use can be faster than
keyboard if trained well and used with discipline.
Spelling Things Out



                                                                  Cinemato-
                               e.                                 graphy by
                        g Mod
                Spellin
                  Julie t Alpha
                             er
                   N ovemb
                                 ra
                 Unifo   rm Sier
                         Zulu




…spelling out words that Dragon doesn’t know and so on. Text use can be faster than
keyboard if trained well and used with discipline.
Spelling Things Out



                                                                  Cinemato-
                               e.      Janusz                     graphy by
                        g Mod
                Spellin
                  Julie t Alpha
                             er
                   N ovemb
                                 ra
                 Unifo   rm Sier
                         Zulu




…spelling out words that Dragon doesn’t know and so on. Text use can be faster than
keyboard if trained well and used with discipline.
Spelling Things Out



                                                                  Cinemato-
                               e.      Janusz                     graphy by
                        g Mod
                Spellin                                           Janusz
                  Julie t Alpha
                             er
                   N ovemb
                                 ra
                 Unifo   rm Sier
                         Zulu




…spelling out words that Dragon doesn’t know and so on. Text use can be faster than
keyboard if trained well and used with discipline.
Using Dragon:
Using the Mouse
Relative Mouse
                        Movement




Dragon provides two modes of mouse control. The first is relative movement, where you
move the mouse in a particular direction.
Relative Mouse
                        Movement




                            e
                mov e mous
                 rig ht four
                    inches




Dragon provides two modes of mouse control. The first is relative movement, where you
move the mouse in a particular direction.
Relative Mouse
                        Movement




                            e
                mov e mous
                 rig ht four
                    inches




Dragon provides two modes of mouse control. The first is relative movement, where you
move the mouse in a particular direction.
Absolute Mouse
                        Movement




The other is the mouse grid, where you progressively choose from within a 3x3 grid, getting
smaller until you reach where you want.
Absolute Mouse
                        Movement




                            .
                  Mouse Grid




The other is the mouse grid, where you progressively choose from within a 3x3 grid, getting
smaller until you reach where you want.
Absolute Mouse
                        Movement




                            .
                  Mouse Grid




The other is the mouse grid, where you progressively choose from within a 3x3 grid, getting
smaller until you reach where you want.
Absolute Mouse
        Movement




      G rid. 6.
Mouse
Absolute Mouse
         Movement




                  .
      G rid. 6. 1
Mouse
Absolute Mouse
                         Movement




                             k.
                  Mou se Clic




Finally you can click or drag or whatever. Mouse use is much slower by voice.
Using Dragon:
Other Features
App-Specific Commands




Dragon has a bunch of commands to make working with common applications easier.
[BUILD]
It’s perhaps worth pointing out that some of the more complex commands never work for
me. But a lot of the simpler ones do fine.
App-Specific Commands



                                               man F rancis.
                                       a rk Nor
                              ail to M          .
                 Sen d an em cks to be you
                         late. Su
                 You’re        essage
                                      .
                            M
                  Se nd the




Dragon has a bunch of commands to make working with common applications easier.
[BUILD]
It’s perhaps worth pointing out that some of the more complex commands never work for
me. But a lot of the simpler ones do fine.
Web Browsing
                        Commands




On Windows there’s some great integration with web browsing, such as speakable form
inputs and links.
Web Browsing
                        Commands




                                                                 Name:




On Windows there’s some great integration with web browsing, such as speakable form
inputs and links.
Web Browsing
                         Commands




                               .
                         name”                                   Name:
                 Click “




On Windows there’s some great integration with web browsing, such as speakable form
inputs and links.
Web Browsing
                          Commands




                                 .
                         name”                                   Name:
                 Click “        t
                  Jame  s Aylet                                  James Aylett




On Windows there’s some great integration with web browsing, such as speakable form
inputs and links.
Custom Commands




You can also define custom commands, although on Windows you need the most expensive
version of DNS. On Mac you can fire keystrokes, AppleScript, shell scripts, Automator
workflows and so on. I use this to make it easier to drive things like Google Reader, which
has a good set of keyboard shortcuts that makes this easier. Generally, if you provide
keyboard shortcuts in webapps I can find a way of using it fairly easily by voice, with a bit of
work.
Implications For
  Web Design
Beware Autoformatting




I talked earlier about Dragon not always knowing when it moves to a different input element
and getting capitalisation wrong. Because of the way it constructs sentences, it can also
insert a stray space at the beginning of a field. This causes a problem with validated
numbers. It’s also a problem with fixed length input elements. Both are big issues with
ecommerce and online banking.
Beware Autoformatting




                                                                    6 digit number:




I talked earlier about Dragon not always knowing when it moves to a different input element
and getting capitalisation wrong. Because of the way it constructs sentences, it can also
insert a stray space at the beginning of a field. This causes a problem with validated
numbers. It’s also a problem with fixed length input elements. Both are big issues with
ecommerce and online banking.
Beware Autoformatting




                                    .                               6 digit number:
                           l 279841
                 N umera                                             279841




I talked earlier about Dragon not always knowing when it moves to a different input element
and getting capitalisation wrong. Because of the way it constructs sentences, it can also
insert a stray space at the beginning of a field. This causes a problem with validated
numbers. It’s also a problem with fixed length input elements. Both are big issues with
ecommerce and online banking.
Don’t Punish My Errors


                                                                  ID:
                                                                  123456

                                                                  ID (repeat):
                                                                  123456

                                                                  PIN:
                                                                  6789

                                                                  PIN (repeat):
                                                                  678




I’m going to make mistakes, either through my own stupidity or through mis-recognition by
Dragon. When that happens, don’t do what Santander’s online banking system does: throw
away validated parts of a form and make me input them again.
Don’t Punish My Errors


                 ID:



                 ID (repeat):



                 PIN:



                 PIN (repeat):
Don’t Punish My Errors


                 ID:



                 ID (repeat):

  *$&£!
                 PIN:



                 PIN (repeat):
Think about Scrolling




If you want to make scrollable content only part of the page then you need to be aware of two
things. Firstly, you MUST focus the thing that should scroll, otherwise hitting Page Down/
Page Up (which is what voice users will do to scroll, effectively) does nothing.
Think about Scrolling



                                                                    Focus the
                                                                  scrollable area




If you want to make scrollable content only part of the page then you need to be aware of two
things. Firstly, you MUST focus the thing that should scroll, otherwise hitting Page Down/
Page Up (which is what voice users will do to scroll, effectively) does nothing.
Think about Scrolling

                 Floating navbar
              A big load of text which is
              going to scroll behind the
              navbar and show how
              annoying it can be when
              people don’t think about
              this carefully.
              A big load of text which is
              going to scroll behind the
              navbar and show how
              annoying it can be when
              people don’t think about
              this carefully.
              A big load of text which is




Secondly, if you overlay anything fixed over the top or bottom of the scrollable content (such
as a navbar, advertising or whatever) then if it’s more than a single line of text high people
scrolling by page are going to MISS CONTENT AND BECOME IRRITATED.
Think about Scrolling

                 Floating navbar
                                            Top of next page will
              A big load of text which is
              going to scroll behind the     hide behind navbar
              navbar and show how
              annoying it can be when
              people don’t think about
              this carefully.
              A big load of text which is
              going to scroll behind the
              navbar and show how
              annoying it can be when
              people don’t think about
              this carefully.
              A big load of text which is




Secondly, if you overlay anything fixed over the top or bottom of the scrollable content (such
as a navbar, advertising or whatever) then if it’s more than a single line of text high people
scrolling by page are going to MISS CONTENT AND BECOME IRRITATED.
Think about Scrolling

                 Floating navbar                                       Floating navbar
                                            Top of next page will
              A big load of text which is                           navbar and show how
              going to scroll behind the     hide behind navbar     annoying it can be when
              navbar and show how                                   people don’t think about
              annoying it can be when                               this carefully.
              people don’t think about                              A big load of text which is
              this carefully.                                       going to scroll behind the
              A big load of text which is                           navbar and show how
              going to scroll behind the                            annoying it can be when
              navbar and show how                                   people don’t think about
              annoying it can be when                               this carefully.
              people don’t think about                              A big load of text which is
              this carefully.                                       going to scroll behind the
              A big load of text which is                           navbar and show how




Secondly, if you overlay anything fixed over the top or bottom of the scrollable content (such
as a navbar, advertising or whatever) then if it’s more than a single line of text high people
scrolling by page are going to MISS CONTENT AND BECOME IRRITATED.
Think about Overlays



                                         Confirm
                                         password:
                                         xxxxxxxxxx




Overlays, little in-page popups to do specific tasks, are almost as trendy as putting navbars
over scrolling content. There are two things you that particularly impact voice users: make
sure Escape closes the current overlay (in one go; some of Twitter’s take two Escapes because
the first just unfocusses the input element), and also MOVE FOCUS to something in the
overlay. If you don’t do the latter, then I have to either use mouse control or tab through all
the elements on the page *behind* the overlay to get to the things you want me to interact
with. (This is all just Good Practice, by the way.)
Think about Overlays


                                                                    Support escape
                                         Confirm                     to close as well
                                         password:
                                         xxxxxxxxxx




Overlays, little in-page popups to do specific tasks, are almost as trendy as putting navbars
over scrolling content. There are two things you that particularly impact voice users: make
sure Escape closes the current overlay (in one go; some of Twitter’s take two Escapes because
the first just unfocusses the input element), and also MOVE FOCUS to something in the
overlay. If you don’t do the latter, then I have to either use mouse control or tab through all
the elements on the page *behind* the overlay to get to the things you want me to interact
with. (This is all just Good Practice, by the way.)
Think about Overlays


                                                                    Support escape
                                         Confirm                     to close as well
    Focus this                           password:
                                         xxxxxxxxxx

  input element




Overlays, little in-page popups to do specific tasks, are almost as trendy as putting navbars
over scrolling content. There are two things you that particularly impact voice users: make
sure Escape closes the current overlay (in one go; some of Twitter’s take two Escapes because
the first just unfocusses the input element), and also MOVE FOCUS to something in the
overlay. If you don’t do the latter, then I have to either use mouse control or tab through all
the elements on the page *behind* the overlay to get to the things you want me to interact
with. (This is all just Good Practice, by the way.)
Beware Focus Events




                                                                           Tweet




This is kind of like overlays. Twitter does a neat thing in its web app: if you haven’t typed into
a tweet reply box, when you blur away (eg by switching application), it collapses the box
again. Please don’t do this.
Beware Focus Events




                            s
                 Open Expres                                               Tweet
                    Editor




This is kind of like overlays. Twitter does a neat thing in its web app: if you haven’t typed into
a tweet reply box, when you blur away (eg by switching application), it collapses the box
again. Please don’t do this.
Beware Focus Events



                                     Open Express
                            s
                 Open Expres            Editor                             Tweet
                    Editor




This is kind of like overlays. Twitter does a neat thing in its web app: if you haven’t typed into
a tweet reply box, when you blur away (eg by switching application), it collapses the box
again. Please don’t do this.
Beware Focus Events




If I’ve opened the Express Editor then the tweet box has disappeared. When I go to transfer
the text back…
Beware Focus Events




                                     Express Editor:




If I’ve opened the Express Editor then the tweet box has disappeared. When I go to transfer
the text back…
Beware Focus Events




                              ah
                 blah blah bl


                                     Express Editor:




If I’ve opened the Express Editor then the tweet box has disappeared. When I go to transfer
the text back…
Beware Focus Events




                                h
                                    blah blah blah
                 blah b lah bla


                                     Express Editor:




If I’ve opened the Express Editor then the tweet box has disappeared. When I go to transfer
the text back…
Beware Focus Events




                                h
                                    blah blah blah
                 blah b lah bla


                                     Express Editor:
                                     blah blah blah


If I’ve opened the Express Editor then the tweet box has disappeared. When I go to transfer
the text back…
Beware Focus Events




                                     t
                                         blah blah blah
                             the tex
                 t ransfer




Nothing is focussed, and the text SIMPLY DISAPPEARS. If I open the Express Editor again, it’s
not there either. With a tweet it’s just frustrating; do this on a serious length of text and it’s
infuriating.
Beware Focus Events
Implications:
                      Autocomplete



This has a few pieces, so I’ve split it out separately.
Support Browser
                     Autocomplete




Some things are painful to dictate; email addresses, for instance. I can shift into spelling
mode, but that takes time. I can type a single character fairly easily however, and then let
browser autocomplete kick in.

So: don’t explicitly disable autocomplete; and GIVE YOUR INPUT ELEMENTS NAMES. Even when
only using Javascript. Also: don’t only use Javascript.
Support Browser
                     Autocomplete



                                                                       Email:




Some things are painful to dictate; email addresses, for instance. I can shift into spelling
mode, but that takes time. I can type a single character fairly easily however, and then let
browser autocomplete kick in.

So: don’t explicitly disable autocomplete; and GIVE YOUR INPUT ELEMENTS NAMES. Even when
only using Javascript. Also: don’t only use Javascript.
Support Browser
                      Autocomplete



                                                                       Email:

                         e key “Juliet”
                        h
                Press t




Some things are painful to dictate; email addresses, for instance. I can shift into spelling
mode, but that takes time. I can type a single character fairly easily however, and then let
browser autocomplete kick in.

So: don’t explicitly disable autocomplete; and GIVE YOUR INPUT ELEMENTS NAMES. Even when
only using Javascript. Also: don’t only use Javascript.
Support Browser
                      Autocomplete



                                                                       Email:
                                                                       j
                         e key “Juliet”
                        h
                Press t                                                james@tartarus.org
                                                                       Name:
                                                                       jaylett@gmail.com
                                                                       james@devfort.com




Some things are painful to dictate; email addresses, for instance. I can shift into spelling
mode, but that takes time. I can type a single character fairly easily however, and then let
browser autocomplete kick in.

So: don’t explicitly disable autocomplete; and GIVE YOUR INPUT ELEMENTS NAMES. Even when
only using Javascript. Also: don’t only use Javascript.
Cautious Application
                 Autocomplete




If I start dictating a name, do NOT autocomplete inline. This is what Mail.app does, and
fortunately I haven’t found a website trying to do the same. The problem is if I mis-speak, or
Dragon incorrectly recognises what I say.
Cautious Application
                 Autocomplete




                                                                     To:




If I start dictating a name, do NOT autocomplete inline. This is what Mail.app does, and
fortunately I haven’t found a website trying to do the same. The problem is if I mis-speak, or
Dragon incorrectly recognises what I say.
Cautious Application
                 Autocomplete




                                                                     To:
                 James




If I start dictating a name, do NOT autocomplete inline. This is what Mail.app does, and
fortunately I haven’t found a website trying to do the same. The problem is if I mis-speak, or
Dragon incorrectly recognises what I say.
Cautious Application
                 Autocomplete




                                                                     To:
                 James                                               James Aylett




If I start dictating a name, do NOT autocomplete inline. This is what Mail.app does, and
fortunately I haven’t found a website trying to do the same. The problem is if I mis-speak, or
Dragon incorrectly recognises what I say.
Cautious Application
                 Autocomplete



                                   Correction options
                                1. James
                                                                      To:
                      t“ James” 2. Chains
                Correc                                                James Aylett
                                3. Jane’s




The problem is that Dragon’s internal cache of the “document” (the field contents) is wrong.
It thinks the input just contains “James”, but the browser (your app via Javascript) has added
another word.
Cautious Application
                 Autocomplete



                                   Correction options
                                 1. James
                                                                      To:
                       2         2. Chains
                Choose                                                James AChains
                                 3. Jane’s




So when Dragon applies the correction, it simply deletes the five characters it put in, and
types new ones in their place. But it’s typing over the automatically added characters, not the
original dictation. Leaving a mess.
But Do Please
         Implement Autocomplete




                                                                      Country:
                                                                      Please select




I hate country lists. The is no correct way of doing a country dropdown: alphabetical and it’s
too long for most people to find their country, out of order and I have no idea what’s going
on (because merely tabbing to a dropdown usually doesn’t open it, and even if it does it’s
often not long enough to guess what’s going on). Worse, I don’t know if you’ve used “United
Kingdom”, “Great Britain”, “UK”, or even “England”. (Three of these are technically correct.)
But Do Please
         Implement Autocomplete




                                                                      Country:
                      ?
                 Erm…                                                 Please select




I hate country lists. The is no correct way of doing a country dropdown: alphabetical and it’s
too long for most people to find their country, out of order and I have no idea what’s going
on (because merely tabbing to a dropdown usually doesn’t open it, and even if it does it’s
often not long enough to guess what’s going on). Worse, I don’t know if you’ve used “United
Kingdom”, “Great Britain”, “UK”, or even “England”. (Three of these are technically correct.)
But Do Please
         Implement Autocomplete




                                                                       Country:
                                                                       Please select




I can open up the dropdown, but unless the entry is near the top (and I can get there by
arrow keys, which are fairly easy to drive by voice) I still have to speak the country. If I say
“United”, then depending on list order sometimes I get “United Kingdom”, and sometimes
“United Arab Emirates” in which case saying “Kingdom” will probably get me to the right
place. If “United” gets me “United Kingdom” and I don’t notice then saying “Kingdom” after a
pause will end up on “Kiribati”. If “United Kingdom” was automatically selected originally and
I didn’t notice, saying “United” can (unexpectedly) wind up selecting “Iran”.
But Do Please
         Implement Autocomplete




                                                                       Country:
                             .
                  Dow n arrow                                          Please select
                                                                       United States
                                                                       United Kingdom
                                                                       Canada




I can open up the dropdown, but unless the entry is near the top (and I can get there by
arrow keys, which are fairly easy to drive by voice) I still have to speak the country. If I say
“United”, then depending on list order sometimes I get “United Kingdom”, and sometimes
“United Arab Emirates” in which case saying “Kingdom” will probably get me to the right
place. If “United” gets me “United Kingdom” and I don’t notice then saying “Kingdom” after a
pause will end up on “Kiribati”. If “United Kingdom” was automatically selected originally and
I didn’t notice, saying “United” can (unexpectedly) wind up selecting “Iran”.
But Do Please
         Implement Autocomplete




If you do a smart autocomplete, I can just dictate the country name in, and then you validate
when I’ve stopped talking (ie you stop receiving keypress / change events). It’s also probably
easier for everyone else in the world who hates long dropdowns. Notably this is what airlines
use these days for selecting airports, and while there are a lot more airports than countries,
there are more than enough of each to justify doing this.
But Do Please
         Implement Autocomplete




                                                                      Country:




If you do a smart autocomplete, I can just dictate the country name in, and then you validate
when I’ve stopped talking (ie you stop receiving keypress / change events). It’s also probably
easier for everyone else in the world who hates long dropdowns. Notably this is what airlines
use these days for selecting airports, and while there are a lot more airports than countries,
there are more than enough of each to justify doing this.
But Do Please
         Implement Autocomplete




                               m                                      Country:
                    ited Kingdo                                       United Kingdom
                 Un




If you do a smart autocomplete, I can just dictate the country name in, and then you validate
when I’ve stopped talking (ie you stop receiving keypress / change events). It’s also probably
easier for everyone else in the world who hates long dropdowns. Notably this is what airlines
use these days for selecting airports, and while there are a lot more airports than countries,
there are more than enough of each to justify doing this.
Live Demo



Hold your breath.
Questions?

              @jaylett
http://bit.ly/accessible-by-voice

More Related Content

Recently uploaded

The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxLoriGlavin3
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxLoriGlavin3
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxLoriGlavin3
 
Training state-of-the-art general text embedding
Training state-of-the-art general text embeddingTraining state-of-the-art general text embedding
Training state-of-the-art general text embeddingZilliz
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxLoriGlavin3
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
What is Artificial Intelligence?????????
What is Artificial Intelligence?????????What is Artificial Intelligence?????????
What is Artificial Intelligence?????????blackmambaettijean
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3
 
Sample pptx for embedding into website for demo
Sample pptx for embedding into website for demoSample pptx for embedding into website for demo
Sample pptx for embedding into website for demoHarshalMandlekar2
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLScyllaDB
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rick Flair
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsPixlogix Infotech
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 

Recently uploaded (20)

The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptx
 
Training state-of-the-art general text embedding
Training state-of-the-art general text embeddingTraining state-of-the-art general text embedding
Training state-of-the-art general text embedding
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
What is Artificial Intelligence?????????
What is Artificial Intelligence?????????What is Artificial Intelligence?????????
What is Artificial Intelligence?????????
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
 
Sample pptx for embedding into website for demo
Sample pptx for embedding into website for demoSample pptx for embedding into website for demo
Sample pptx for embedding into website for demo
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQL
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and Cons
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 

Featured

2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by HubspotMarius Sescu
 
Everything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPTEverything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPTExpeed Software
 
Product Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage EngineeringsProduct Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage EngineeringsPixeldarts
 
How Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental HealthHow Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental HealthThinkNow
 
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdfAI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdfmarketingartwork
 
PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024Neil Kimberley
 
Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)contently
 
How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024Albert Qian
 
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsSocial Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsKurio // The Social Media Age(ncy)
 
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Search Engine Journal
 
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summarySpeakerHub
 
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd Clark Boyd
 
Getting into the tech field. what next
Getting into the tech field. what next Getting into the tech field. what next
Getting into the tech field. what next Tessa Mero
 
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentGoogle's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentLily Ray
 
Time Management & Productivity - Best Practices
Time Management & Productivity -  Best PracticesTime Management & Productivity -  Best Practices
Time Management & Productivity - Best PracticesVit Horky
 
The six step guide to practical project management
The six step guide to practical project managementThe six step guide to practical project management
The six step guide to practical project managementMindGenius
 
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...RachelPearson36
 

Featured (20)

2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot
 
Everything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPTEverything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPT
 
Product Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage EngineeringsProduct Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage Engineerings
 
How Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental HealthHow Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental Health
 
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdfAI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
 
Skeleton Culture Code
Skeleton Culture CodeSkeleton Culture Code
Skeleton Culture Code
 
PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024
 
Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)
 
How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024
 
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsSocial Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie Insights
 
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024
 
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary
 
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd
 
Getting into the tech field. what next
Getting into the tech field. what next Getting into the tech field. what next
Getting into the tech field. what next
 
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentGoogle's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search Intent
 
How to have difficult conversations
How to have difficult conversations How to have difficult conversations
How to have difficult conversations
 
Introduction to Data Science
Introduction to Data ScienceIntroduction to Data Science
Introduction to Data Science
 
Time Management & Productivity - Best Practices
Time Management & Productivity -  Best PracticesTime Management & Productivity -  Best Practices
Time Management & Productivity - Best Practices
 
The six step guide to practical project management
The six step guide to practical project managementThe six step guide to practical project management
The six step guide to practical project management
 
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
 

Accessible by Voice

  • 1. Accessible by Voice Or: how to make a tiny corner of the internet not want to punch you in the face
  • 2. Talking to the Dragon • Dragon Naturally Speaking Home £79.99 • Dragon Naturally Speaking Premium £149.99 • Dragon Naturally Speaking Professional £549.00 • Dragon Dictate for Mac £129.99 (I don’t make any money off Dragon. I wish I did ;-) There are others, including dictation software built into Mac OS & Windows, and rudimentary dictation / control software built into Android, iOS and Windows Phone 7. There are also some highly specialised systems (eg: medical), and a few other dictation-only solutions as well. As far as I’m aware, Dragon is the only system that has reasonable voice control of desktop computers as well as dictation capabilities.
  • 4. Text input So how does it work? You talk into a microphone, Dragon does some magic (involving maths), and text appears in the target application. But that’s just transcription; what if you want to correct what you dictated?
  • 5. Text input h a h bla Blah bl So how does it work? You talk into a microphone, Dragon does some magic (involving maths), and text appears in the target application. But that’s just transcription; what if you want to correct what you dictated?
  • 6. Text input Blah blah blah h a h bla Blah bl So how does it work? You talk into a microphone, Dragon does some magic (involving maths), and text appears in the target application. But that’s just transcription; what if you want to correct what you dictated?
  • 7. Editing Dragon keeps an internal representation of the text that has been dictated.
  • 8. Editing sh Wh en I wi ud upo n a clo Dragon keeps an internal representation of the text that has been dictated.
  • 9. Editing When I wish sh Wh en I wi upon a cloud ud upo n a clo Dragon keeps an internal representation of the text that has been dictated.
  • 10. Editing When I When I wish wish upon a sh Wh en I wi upon a cloud upo n a clo ud cloud Dragon keeps an internal representation of the text that has been dictated.
  • 11. Editing When I When I wish wish upon a Delete “wish” upon a cloud cloud It applies voice editing commands to its internal representation, and *then* issues editing commands to the target application.
  • 12. Editing When I When I dance dance upon dance upon a cloud a cloud To do this, it keeps track of where the text cursor is in the target application.
  • 13. Editing When I When I dance I dance I t ocumen dance for me Cache d dance for me If things get out of sync (if you use the keyboard to edit, for instance) you can ask Dragon to re-read what’s in the application. This is also useful when you load a new document.
  • 14. Editing Sometimes this goes wrong, particularly with apps that haven’t properly implemented accessibility. Good apps on Mac: Textmate, Scrivener. Bad: Chrome. Somewhere in the middle: Mail. Things that can go wrong range from random letters appearing, to overwriting parts of words. Sometimes Dragon can’t even get back in sync with the application.
  • 15. Editing h Blah b lah bla Sometimes this goes wrong, particularly with apps that haven’t properly implemented accessibility. Good apps on Mac: Textmate, Scrivener. Bad: Chrome. Somewhere in the middle: Mail. Things that can go wrong range from random letters appearing, to overwriting parts of words. Sometimes Dragon can’t even get back in sync with the application.
  • 16. Editing h Blah blah blah Blah b lah bla Sometimes this goes wrong, particularly with apps that haven’t properly implemented accessibility. Good apps on Mac: Textmate, Scrivener. Bad: Chrome. Somewhere in the middle: Mail. Things that can go wrong range from random letters appearing, to overwriting parts of words. Sometimes Dragon can’t even get back in sync with the application.
  • 17. Editing Blah blah blah Blah blah h Blah b lah bla blahJ Sometimes this goes wrong, particularly with apps that haven’t properly implemented accessibility. Good apps on Mac: Textmate, Scrivener. Bad: Chrome. Somewhere in the middle: Mail. Things that can go wrong range from random letters appearing, to overwriting parts of words. Sometimes Dragon can’t even get back in sync with the application.
  • 18. Automatic Capitalisation Dragon is designed (mostly) for dictating prose, so it understands that new sentences get capital letters. If it doesn’t notice that you’ve moved from one input to the next, it may not think you’re in a new sentence. (Conversely, sometimes it thinks you’re at the start of a new sentence but you don’t want to be.) There are commands to force or suppress capitalisation of the next word.
  • 19. Automatic Capitalisation o Iw anted t u m ak e y o happy Dragon is designed (mostly) for dictating prose, so it understands that new sentences get capital letters. If it doesn’t notice that you’ve moved from one input to the next, it may not think you’re in a new sentence. (Conversely, sometimes it thinks you’re at the start of a new sentence but you don’t want to be.) There are commands to force or suppress capitalisation of the next word.
  • 20. Automatic Capitalisation I wanted to o Iw anted t make you happy u m ak e y o happy Dragon is designed (mostly) for dictating prose, so it understands that new sentences get capital letters. If it doesn’t notice that you’ve moved from one input to the next, it may not think you’re in a new sentence. (Conversely, sometimes it thinks you’re at the start of a new sentence but you don’t want to be.) There are commands to force or suppress capitalisation of the next word.
  • 21. Automatic Capitalisation Field 1: I wanted to I wanted to o Iw anted t make you happy u m ak e y o Field 2: happy make you happy Dragon is designed (mostly) for dictating prose, so it understands that new sentences get capital letters. If it doesn’t notice that you’ve moved from one input to the next, it may not think you’re in a new sentence. (Conversely, sometimes it thinks you’re at the start of a new sentence but you don’t want to be.) There are commands to force or suppress capitalisation of the next word.
  • 22. Express Editor Dragon features an Express Editor which allows you to create a piece of text, editing as you go, then insert it at the end. More tightly bound to Dragon, it makes fewer mistakes.
  • 23. Express Editor ah blah blah bl Dragon features an Express Editor which allows you to create a piece of text, editing as you go, then insert it at the end. More tightly bound to Dragon, it makes fewer mistakes.
  • 24. Express Editor h blah blah blah blah b lah bla Dragon features an Express Editor which allows you to create a piece of text, editing as you go, then insert it at the end. More tightly bound to Dragon, it makes fewer mistakes.
  • 25. Express Editor h blah blah blah blah b lah bla Express Editor: blah blah blah Dragon features an Express Editor which allows you to create a piece of text, editing as you go, then insert it at the end. More tightly bound to Dragon, it makes fewer mistakes.
  • 26. Express Editor blah blah blah Once it’s transferred, if you need to do further editing you can cache it back into Dragon.
  • 27. Express Editor t blah blah blah the tex t ransfer Once it’s transferred, if you need to do further editing you can cache it back into Dragon.
  • 28. Express Editor Text field: t blah blah blah blah blah the tex t ransfer blah Once it’s transferred, if you need to do further editing you can cache it back into Dragon.
  • 29. Correcting Dictation Ye Highlands and ye Lowlands, Oh, where hae ye been? They hae slain the Earl O' Moray, And Lady Mondegreen. * Sylvia Wright (1954). “The Death of Lady Mondegreen”. Harper’s Magazine 209 (1254): 48-51. Dragon also has features for correcting dictation errors…
  • 30. Correcting Dictation Ye Highlands and ye Lowlands, Oh, where hae ye been? d ct “An They hae slain the Earl corre O' Moray, Lady And Lady Mondegreen. n” Mon degree * Sylvia Wright (1954). “The Death of Lady Mondegreen”. Harper’s Magazine 209 (1254): 48-51. Dragon also has features for correcting dictation errors…
  • 31. Correcting Dictation Ye Highlands and ye Correction options Lowlands, 1. And Lady Mondegreen Oh, where hae ye been? d ct “An They hae slain the Earl 2. And laid him on the green corre O' Moray, Lady 3. and lady mondegreen And Lady Mondegreen. n” Mon degree * Sylvia Wright (1954). “The Death of Lady Mondegreen”. Harper’s Magazine 209 (1254): 48-51. Dragon also has features for correcting dictation errors…
  • 32. Spelling Things Out …spelling out words that Dragon doesn’t know and so on. Text use can be faster than keyboard if trained well and used with discipline.
  • 33. Spelling Things Out Cinemato- graphy by …spelling out words that Dragon doesn’t know and so on. Text use can be faster than keyboard if trained well and used with discipline.
  • 34. Spelling Things Out Cinemato- e. graphy by g Mod Spellin Julie t Alpha er N ovemb ra Unifo rm Sier Zulu …spelling out words that Dragon doesn’t know and so on. Text use can be faster than keyboard if trained well and used with discipline.
  • 35. Spelling Things Out Cinemato- e. Janusz graphy by g Mod Spellin Julie t Alpha er N ovemb ra Unifo rm Sier Zulu …spelling out words that Dragon doesn’t know and so on. Text use can be faster than keyboard if trained well and used with discipline.
  • 36. Spelling Things Out Cinemato- e. Janusz graphy by g Mod Spellin Janusz Julie t Alpha er N ovemb ra Unifo rm Sier Zulu …spelling out words that Dragon doesn’t know and so on. Text use can be faster than keyboard if trained well and used with discipline.
  • 38. Relative Mouse Movement Dragon provides two modes of mouse control. The first is relative movement, where you move the mouse in a particular direction.
  • 39. Relative Mouse Movement e mov e mous rig ht four inches Dragon provides two modes of mouse control. The first is relative movement, where you move the mouse in a particular direction.
  • 40. Relative Mouse Movement e mov e mous rig ht four inches Dragon provides two modes of mouse control. The first is relative movement, where you move the mouse in a particular direction.
  • 41. Absolute Mouse Movement The other is the mouse grid, where you progressively choose from within a 3x3 grid, getting smaller until you reach where you want.
  • 42. Absolute Mouse Movement . Mouse Grid The other is the mouse grid, where you progressively choose from within a 3x3 grid, getting smaller until you reach where you want.
  • 43. Absolute Mouse Movement . Mouse Grid The other is the mouse grid, where you progressively choose from within a 3x3 grid, getting smaller until you reach where you want.
  • 44. Absolute Mouse Movement G rid. 6. Mouse
  • 45. Absolute Mouse Movement . G rid. 6. 1 Mouse
  • 46. Absolute Mouse Movement k. Mou se Clic Finally you can click or drag or whatever. Mouse use is much slower by voice.
  • 48. App-Specific Commands Dragon has a bunch of commands to make working with common applications easier. [BUILD] It’s perhaps worth pointing out that some of the more complex commands never work for me. But a lot of the simpler ones do fine.
  • 49. App-Specific Commands man F rancis. a rk Nor ail to M . Sen d an em cks to be you late. Su You’re essage . M Se nd the Dragon has a bunch of commands to make working with common applications easier. [BUILD] It’s perhaps worth pointing out that some of the more complex commands never work for me. But a lot of the simpler ones do fine.
  • 50. Web Browsing Commands On Windows there’s some great integration with web browsing, such as speakable form inputs and links.
  • 51. Web Browsing Commands Name: On Windows there’s some great integration with web browsing, such as speakable form inputs and links.
  • 52. Web Browsing Commands . name” Name: Click “ On Windows there’s some great integration with web browsing, such as speakable form inputs and links.
  • 53. Web Browsing Commands . name” Name: Click “ t Jame s Aylet James Aylett On Windows there’s some great integration with web browsing, such as speakable form inputs and links.
  • 54. Custom Commands You can also define custom commands, although on Windows you need the most expensive version of DNS. On Mac you can fire keystrokes, AppleScript, shell scripts, Automator workflows and so on. I use this to make it easier to drive things like Google Reader, which has a good set of keyboard shortcuts that makes this easier. Generally, if you provide keyboard shortcuts in webapps I can find a way of using it fairly easily by voice, with a bit of work.
  • 55. Implications For Web Design
  • 56. Beware Autoformatting I talked earlier about Dragon not always knowing when it moves to a different input element and getting capitalisation wrong. Because of the way it constructs sentences, it can also insert a stray space at the beginning of a field. This causes a problem with validated numbers. It’s also a problem with fixed length input elements. Both are big issues with ecommerce and online banking.
  • 57. Beware Autoformatting 6 digit number: I talked earlier about Dragon not always knowing when it moves to a different input element and getting capitalisation wrong. Because of the way it constructs sentences, it can also insert a stray space at the beginning of a field. This causes a problem with validated numbers. It’s also a problem with fixed length input elements. Both are big issues with ecommerce and online banking.
  • 58. Beware Autoformatting . 6 digit number: l 279841 N umera 279841 I talked earlier about Dragon not always knowing when it moves to a different input element and getting capitalisation wrong. Because of the way it constructs sentences, it can also insert a stray space at the beginning of a field. This causes a problem with validated numbers. It’s also a problem with fixed length input elements. Both are big issues with ecommerce and online banking.
  • 59. Don’t Punish My Errors ID: 123456 ID (repeat): 123456 PIN: 6789 PIN (repeat): 678 I’m going to make mistakes, either through my own stupidity or through mis-recognition by Dragon. When that happens, don’t do what Santander’s online banking system does: throw away validated parts of a form and make me input them again.
  • 60. Don’t Punish My Errors ID: ID (repeat): PIN: PIN (repeat):
  • 61. Don’t Punish My Errors ID: ID (repeat): *$&£! PIN: PIN (repeat):
  • 62. Think about Scrolling If you want to make scrollable content only part of the page then you need to be aware of two things. Firstly, you MUST focus the thing that should scroll, otherwise hitting Page Down/ Page Up (which is what voice users will do to scroll, effectively) does nothing.
  • 63. Think about Scrolling Focus the scrollable area If you want to make scrollable content only part of the page then you need to be aware of two things. Firstly, you MUST focus the thing that should scroll, otherwise hitting Page Down/ Page Up (which is what voice users will do to scroll, effectively) does nothing.
  • 64. Think about Scrolling Floating navbar A big load of text which is going to scroll behind the navbar and show how annoying it can be when people don’t think about this carefully. A big load of text which is going to scroll behind the navbar and show how annoying it can be when people don’t think about this carefully. A big load of text which is Secondly, if you overlay anything fixed over the top or bottom of the scrollable content (such as a navbar, advertising or whatever) then if it’s more than a single line of text high people scrolling by page are going to MISS CONTENT AND BECOME IRRITATED.
  • 65. Think about Scrolling Floating navbar Top of next page will A big load of text which is going to scroll behind the hide behind navbar navbar and show how annoying it can be when people don’t think about this carefully. A big load of text which is going to scroll behind the navbar and show how annoying it can be when people don’t think about this carefully. A big load of text which is Secondly, if you overlay anything fixed over the top or bottom of the scrollable content (such as a navbar, advertising or whatever) then if it’s more than a single line of text high people scrolling by page are going to MISS CONTENT AND BECOME IRRITATED.
  • 66. Think about Scrolling Floating navbar Floating navbar Top of next page will A big load of text which is navbar and show how going to scroll behind the hide behind navbar annoying it can be when navbar and show how people don’t think about annoying it can be when this carefully. people don’t think about A big load of text which is this carefully. going to scroll behind the A big load of text which is navbar and show how going to scroll behind the annoying it can be when navbar and show how people don’t think about annoying it can be when this carefully. people don’t think about A big load of text which is this carefully. going to scroll behind the A big load of text which is navbar and show how Secondly, if you overlay anything fixed over the top or bottom of the scrollable content (such as a navbar, advertising or whatever) then if it’s more than a single line of text high people scrolling by page are going to MISS CONTENT AND BECOME IRRITATED.
  • 67. Think about Overlays Confirm password: xxxxxxxxxx Overlays, little in-page popups to do specific tasks, are almost as trendy as putting navbars over scrolling content. There are two things you that particularly impact voice users: make sure Escape closes the current overlay (in one go; some of Twitter’s take two Escapes because the first just unfocusses the input element), and also MOVE FOCUS to something in the overlay. If you don’t do the latter, then I have to either use mouse control or tab through all the elements on the page *behind* the overlay to get to the things you want me to interact with. (This is all just Good Practice, by the way.)
  • 68. Think about Overlays Support escape Confirm to close as well password: xxxxxxxxxx Overlays, little in-page popups to do specific tasks, are almost as trendy as putting navbars over scrolling content. There are two things you that particularly impact voice users: make sure Escape closes the current overlay (in one go; some of Twitter’s take two Escapes because the first just unfocusses the input element), and also MOVE FOCUS to something in the overlay. If you don’t do the latter, then I have to either use mouse control or tab through all the elements on the page *behind* the overlay to get to the things you want me to interact with. (This is all just Good Practice, by the way.)
  • 69. Think about Overlays Support escape Confirm to close as well Focus this password: xxxxxxxxxx input element Overlays, little in-page popups to do specific tasks, are almost as trendy as putting navbars over scrolling content. There are two things you that particularly impact voice users: make sure Escape closes the current overlay (in one go; some of Twitter’s take two Escapes because the first just unfocusses the input element), and also MOVE FOCUS to something in the overlay. If you don’t do the latter, then I have to either use mouse control or tab through all the elements on the page *behind* the overlay to get to the things you want me to interact with. (This is all just Good Practice, by the way.)
  • 70. Beware Focus Events Tweet This is kind of like overlays. Twitter does a neat thing in its web app: if you haven’t typed into a tweet reply box, when you blur away (eg by switching application), it collapses the box again. Please don’t do this.
  • 71. Beware Focus Events s Open Expres Tweet Editor This is kind of like overlays. Twitter does a neat thing in its web app: if you haven’t typed into a tweet reply box, when you blur away (eg by switching application), it collapses the box again. Please don’t do this.
  • 72. Beware Focus Events Open Express s Open Expres Editor Tweet Editor This is kind of like overlays. Twitter does a neat thing in its web app: if you haven’t typed into a tweet reply box, when you blur away (eg by switching application), it collapses the box again. Please don’t do this.
  • 73. Beware Focus Events If I’ve opened the Express Editor then the tweet box has disappeared. When I go to transfer the text back…
  • 74. Beware Focus Events Express Editor: If I’ve opened the Express Editor then the tweet box has disappeared. When I go to transfer the text back…
  • 75. Beware Focus Events ah blah blah bl Express Editor: If I’ve opened the Express Editor then the tweet box has disappeared. When I go to transfer the text back…
  • 76. Beware Focus Events h blah blah blah blah b lah bla Express Editor: If I’ve opened the Express Editor then the tweet box has disappeared. When I go to transfer the text back…
  • 77. Beware Focus Events h blah blah blah blah b lah bla Express Editor: blah blah blah If I’ve opened the Express Editor then the tweet box has disappeared. When I go to transfer the text back…
  • 78. Beware Focus Events t blah blah blah the tex t ransfer Nothing is focussed, and the text SIMPLY DISAPPEARS. If I open the Express Editor again, it’s not there either. With a tweet it’s just frustrating; do this on a serious length of text and it’s infuriating.
  • 80. Implications: Autocomplete This has a few pieces, so I’ve split it out separately.
  • 81. Support Browser Autocomplete Some things are painful to dictate; email addresses, for instance. I can shift into spelling mode, but that takes time. I can type a single character fairly easily however, and then let browser autocomplete kick in. So: don’t explicitly disable autocomplete; and GIVE YOUR INPUT ELEMENTS NAMES. Even when only using Javascript. Also: don’t only use Javascript.
  • 82. Support Browser Autocomplete Email: Some things are painful to dictate; email addresses, for instance. I can shift into spelling mode, but that takes time. I can type a single character fairly easily however, and then let browser autocomplete kick in. So: don’t explicitly disable autocomplete; and GIVE YOUR INPUT ELEMENTS NAMES. Even when only using Javascript. Also: don’t only use Javascript.
  • 83. Support Browser Autocomplete Email: e key “Juliet” h Press t Some things are painful to dictate; email addresses, for instance. I can shift into spelling mode, but that takes time. I can type a single character fairly easily however, and then let browser autocomplete kick in. So: don’t explicitly disable autocomplete; and GIVE YOUR INPUT ELEMENTS NAMES. Even when only using Javascript. Also: don’t only use Javascript.
  • 84. Support Browser Autocomplete Email: j e key “Juliet” h Press t james@tartarus.org Name: jaylett@gmail.com james@devfort.com Some things are painful to dictate; email addresses, for instance. I can shift into spelling mode, but that takes time. I can type a single character fairly easily however, and then let browser autocomplete kick in. So: don’t explicitly disable autocomplete; and GIVE YOUR INPUT ELEMENTS NAMES. Even when only using Javascript. Also: don’t only use Javascript.
  • 85. Cautious Application Autocomplete If I start dictating a name, do NOT autocomplete inline. This is what Mail.app does, and fortunately I haven’t found a website trying to do the same. The problem is if I mis-speak, or Dragon incorrectly recognises what I say.
  • 86. Cautious Application Autocomplete To: If I start dictating a name, do NOT autocomplete inline. This is what Mail.app does, and fortunately I haven’t found a website trying to do the same. The problem is if I mis-speak, or Dragon incorrectly recognises what I say.
  • 87. Cautious Application Autocomplete To: James If I start dictating a name, do NOT autocomplete inline. This is what Mail.app does, and fortunately I haven’t found a website trying to do the same. The problem is if I mis-speak, or Dragon incorrectly recognises what I say.
  • 88. Cautious Application Autocomplete To: James James Aylett If I start dictating a name, do NOT autocomplete inline. This is what Mail.app does, and fortunately I haven’t found a website trying to do the same. The problem is if I mis-speak, or Dragon incorrectly recognises what I say.
  • 89. Cautious Application Autocomplete Correction options 1. James To: t“ James” 2. Chains Correc James Aylett 3. Jane’s The problem is that Dragon’s internal cache of the “document” (the field contents) is wrong. It thinks the input just contains “James”, but the browser (your app via Javascript) has added another word.
  • 90. Cautious Application Autocomplete Correction options 1. James To: 2 2. Chains Choose James AChains 3. Jane’s So when Dragon applies the correction, it simply deletes the five characters it put in, and types new ones in their place. But it’s typing over the automatically added characters, not the original dictation. Leaving a mess.
  • 91. But Do Please Implement Autocomplete Country: Please select I hate country lists. The is no correct way of doing a country dropdown: alphabetical and it’s too long for most people to find their country, out of order and I have no idea what’s going on (because merely tabbing to a dropdown usually doesn’t open it, and even if it does it’s often not long enough to guess what’s going on). Worse, I don’t know if you’ve used “United Kingdom”, “Great Britain”, “UK”, or even “England”. (Three of these are technically correct.)
  • 92. But Do Please Implement Autocomplete Country: ? Erm… Please select I hate country lists. The is no correct way of doing a country dropdown: alphabetical and it’s too long for most people to find their country, out of order and I have no idea what’s going on (because merely tabbing to a dropdown usually doesn’t open it, and even if it does it’s often not long enough to guess what’s going on). Worse, I don’t know if you’ve used “United Kingdom”, “Great Britain”, “UK”, or even “England”. (Three of these are technically correct.)
  • 93. But Do Please Implement Autocomplete Country: Please select I can open up the dropdown, but unless the entry is near the top (and I can get there by arrow keys, which are fairly easy to drive by voice) I still have to speak the country. If I say “United”, then depending on list order sometimes I get “United Kingdom”, and sometimes “United Arab Emirates” in which case saying “Kingdom” will probably get me to the right place. If “United” gets me “United Kingdom” and I don’t notice then saying “Kingdom” after a pause will end up on “Kiribati”. If “United Kingdom” was automatically selected originally and I didn’t notice, saying “United” can (unexpectedly) wind up selecting “Iran”.
  • 94. But Do Please Implement Autocomplete Country: . Dow n arrow Please select United States United Kingdom Canada I can open up the dropdown, but unless the entry is near the top (and I can get there by arrow keys, which are fairly easy to drive by voice) I still have to speak the country. If I say “United”, then depending on list order sometimes I get “United Kingdom”, and sometimes “United Arab Emirates” in which case saying “Kingdom” will probably get me to the right place. If “United” gets me “United Kingdom” and I don’t notice then saying “Kingdom” after a pause will end up on “Kiribati”. If “United Kingdom” was automatically selected originally and I didn’t notice, saying “United” can (unexpectedly) wind up selecting “Iran”.
  • 95. But Do Please Implement Autocomplete If you do a smart autocomplete, I can just dictate the country name in, and then you validate when I’ve stopped talking (ie you stop receiving keypress / change events). It’s also probably easier for everyone else in the world who hates long dropdowns. Notably this is what airlines use these days for selecting airports, and while there are a lot more airports than countries, there are more than enough of each to justify doing this.
  • 96. But Do Please Implement Autocomplete Country: If you do a smart autocomplete, I can just dictate the country name in, and then you validate when I’ve stopped talking (ie you stop receiving keypress / change events). It’s also probably easier for everyone else in the world who hates long dropdowns. Notably this is what airlines use these days for selecting airports, and while there are a lot more airports than countries, there are more than enough of each to justify doing this.
  • 97. But Do Please Implement Autocomplete m Country: ited Kingdo United Kingdom Un If you do a smart autocomplete, I can just dictate the country name in, and then you validate when I’ve stopped talking (ie you stop receiving keypress / change events). It’s also probably easier for everyone else in the world who hates long dropdowns. Notably this is what airlines use these days for selecting airports, and while there are a lot more airports than countries, there are more than enough of each to justify doing this.
  • 99. Questions? @jaylett http://bit.ly/accessible-by-voice