2013 Speech TEK - Alphanumeric Recognition Discussion

Crispin Reedy
Crispin ReedyVoice User Experience Designer at Versay
© 2002 – 2012 Versay Solutions, LLC. All rights reserved.
Alphanumeric Speech
Recognition
SpeechTek
August 19, 2013
Crispin Reedy
“The fault, dear Brutus, is not in
our stars, but in ourselves”
-- Julius Caesar, Act I, scene ii
2
The ProblemWithAlphanumerics
© 2002 – 2012 Versay Solutions, LLC. All rights reserved.
The Need
• Account Numbers
• Policy Numbers
• Spelling out names and addresses
• Special cases
– VIN, Canadian Postal Code
• And more…
3
© 2002 – 2012 Versay Solutions, LLC. All rights reserved.
Methods for Addressing
• Project Tactics
• Limit the grammar
– Constraint List
– N-Best + Back-End Data Validation
• Confirmation
• Prefiller
4
© 2002 – 2012 Versay Solutions, LLC. All rights reserved.
Project Tactics
• Can you avoid it?
– Phone number / SSN / Zip / DOB?
• Set expectations
– Not always easy!
• Describe the problem
• What tools do you have available?
– Constraints / patterns?
– Back-end data source available?
• Can you run a proof of concept / experiment?
5
© 2002 – 2012 Versay Solutions, LLC. All rights reserved.
Constraints and Patterns
• Does the number have any known pattern
that can be used to limit possible values (and
thereby improve recognition)
– For example:
• First character is always A
• First three characters are always numbers
• Last characters are always C, G or T.
• If the answer is “no,” consider doing your own
analysis.
– Even if you don’t think there is a pattern, there
may be one.
6
© 2002 – 2012 Versay Solutions, LLC. All rights reserved.
Applying Constraints
• Writing grammar specifically for the pattern
– How complicated is it?
• Applying a constraint list.
– How big is it?
7
© 2002 – 2012 Versay Solutions, LLC. All rights reserved.
Using nBest + Back-End Data
• Collect using an unconstrained grammar
• Set your recognizer to return an nBest list.
• Use a webservice / back end data dip to
determine which ones are “real.”
• Confirm the first “real” one on the list
– Throw out the ones that are not real.
• If no, confirm the second “real” one on the
list.
– Potentially collect again after that.
8
© 2002 – 2012 Versay Solutions, LLC. All rights reserved.
Confirmation Strategy
• PROTIP: Phonemes that are difficult for the
recognizer to hear … are also difficult for
humans to hear when they are spoken back.
• Confirm using letter names for easily
confusable alphanumerics.
– “You said 8, 2, 7 G as in George, B as in Boy, 9. Is
that right?”
9
© 2002 – 2012 Versay Solutions, LLC. All rights reserved.
What About Letter Names?
• Yes with caveats:
– Do you have a special domain that would allow
you to teach the caller letter names?
– Letter names invented by the caller will be quite
variable.
• Some of the “oddballs” will never be recognized
– If letter names are used during confirmation, and
the utterance is re-collected, the caller may tend
to use those letter names during the second
collection.
• So add them.
10
© 2002 – 2012 Versay Solutions, LLC. All rights reserved.
What About Letter Names?
• Yes, because:
– Longer utterances “B as in Boy” are not likely to
generate false acceptance between shorter
utterances such as “G” “T” etc.
• Make them separate rules so they can be
weighted
11
© 2002 – 2012 Versay Solutions, LLC. All rights reserved.
Using Prefiller
• “The account number is… B Z 3 9 0”
– Noticeable improvement in recognition of first
letter
– Caller may spontaneously offer
– Consider teaching the caller to say the prefiller
• Especially if you have repeat callers
12
© 2002 – 2012 Versay Solutions, LLC. All rights reserved.
Other Suggestions
• Look at speech recognition parameters that
are not directly related to alphanumeric
– Are callers calling from a very noisy environment?
• Adjust overall speech threshold
– Timing of utterance collection?
• Listen to recording of utterances to make sure
everything is getting collected
13
© 2002 – 2012 Versay Solutions, LLC. All rights reserved.
Specific Cases
• VIN
– Has specific pattern, but different for each
manufacturer
– 16 digits: nobody will want to re-enter if you get it
wrong.
14
IT DEPENDS!
15
but which way is “the best?”
1 of 15

Recommended

e7copper by
e7coppere7copper
e7copperRobert Priboth
46 views1 slide
Design Thinking Action Lab Exercise 1 by
Design Thinking Action Lab Exercise 1Design Thinking Action Lab Exercise 1
Design Thinking Action Lab Exercise 1Crispin Reedy
327 views1 slide
Association for Voice Interaction Design - Annual Meeting 2018 by
Association for Voice Interaction Design - Annual Meeting 2018Association for Voice Interaction Design - Annual Meeting 2018
Association for Voice Interaction Design - Annual Meeting 2018Crispin Reedy
389 views15 slides
Assertive Niceness by
Assertive NicenessAssertive Niceness
Assertive NicenessCrispin Reedy
600 views48 slides
Voice User Interface Design - Big Design 2017 by
Voice User Interface Design - Big Design 2017Voice User Interface Design - Big Design 2017
Voice User Interface Design - Big Design 2017Crispin Reedy
2.7K views76 slides
Conversational User Interfaces, Past and Future by
Conversational User Interfaces, Past and FutureConversational User Interfaces, Past and Future
Conversational User Interfaces, Past and FutureCrispin Reedy
1.9K views58 slides

More Related Content

More from Crispin Reedy

Association for Voice Interaction Design Annual Meeting 2017 by
Association for Voice Interaction Design Annual Meeting 2017Association for Voice Interaction Design Annual Meeting 2017
Association for Voice Interaction Design Annual Meeting 2017Crispin Reedy
486 views15 slides
Where's Jarvis? The Future of Voice Recognition and Natural Language User In... by
Where's Jarvis?  The Future of Voice Recognition and Natural Language User In...Where's Jarvis?  The Future of Voice Recognition and Natural Language User In...
Where's Jarvis? The Future of Voice Recognition and Natural Language User In...Crispin Reedy
974 views77 slides
Voice Recognition and Natural Language - Dallas TechFest 2016 by
Voice Recognition and Natural Language - Dallas TechFest 2016Voice Recognition and Natural Language - Dallas TechFest 2016
Voice Recognition and Natural Language - Dallas TechFest 2016Crispin Reedy
1K views88 slides
Top 10 Tips for Making Complicated Things Simple by
Top 10 Tips for Making Complicated Things SimpleTop 10 Tips for Making Complicated Things Simple
Top 10 Tips for Making Complicated Things SimpleCrispin Reedy
7.8K views91 slides
Association for Voice Interaction Design Annual Meeting 2016 by
Association for Voice Interaction Design Annual Meeting 2016Association for Voice Interaction Design Annual Meeting 2016
Association for Voice Interaction Design Annual Meeting 2016Crispin Reedy
431 views17 slides
Going Solo: Design and Productivity Techniques for the Team of One by
Going Solo: Design and Productivity Techniques for the Team of OneGoing Solo: Design and Productivity Techniques for the Team of One
Going Solo: Design and Productivity Techniques for the Team of OneCrispin Reedy
645 views81 slides

More from Crispin Reedy(9)

Association for Voice Interaction Design Annual Meeting 2017 by Crispin Reedy
Association for Voice Interaction Design Annual Meeting 2017Association for Voice Interaction Design Annual Meeting 2017
Association for Voice Interaction Design Annual Meeting 2017
Crispin Reedy486 views
Where's Jarvis? The Future of Voice Recognition and Natural Language User In... by Crispin Reedy
Where's Jarvis?  The Future of Voice Recognition and Natural Language User In...Where's Jarvis?  The Future of Voice Recognition and Natural Language User In...
Where's Jarvis? The Future of Voice Recognition and Natural Language User In...
Crispin Reedy974 views
Voice Recognition and Natural Language - Dallas TechFest 2016 by Crispin Reedy
Voice Recognition and Natural Language - Dallas TechFest 2016Voice Recognition and Natural Language - Dallas TechFest 2016
Voice Recognition and Natural Language - Dallas TechFest 2016
Crispin Reedy1K views
Top 10 Tips for Making Complicated Things Simple by Crispin Reedy
Top 10 Tips for Making Complicated Things SimpleTop 10 Tips for Making Complicated Things Simple
Top 10 Tips for Making Complicated Things Simple
Crispin Reedy7.8K views
Association for Voice Interaction Design Annual Meeting 2016 by Crispin Reedy
Association for Voice Interaction Design Annual Meeting 2016Association for Voice Interaction Design Annual Meeting 2016
Association for Voice Interaction Design Annual Meeting 2016
Crispin Reedy431 views
Going Solo: Design and Productivity Techniques for the Team of One by Crispin Reedy
Going Solo: Design and Productivity Techniques for the Team of OneGoing Solo: Design and Productivity Techniques for the Team of One
Going Solo: Design and Productivity Techniques for the Team of One
Crispin Reedy645 views
Service Design and the Omnichannel Experience - SpeechTEK 2015 by Crispin Reedy
Service Design and the Omnichannel Experience - SpeechTEK 2015Service Design and the Omnichannel Experience - SpeechTEK 2015
Service Design and the Omnichannel Experience - SpeechTEK 2015
Crispin Reedy4.9K views
Association for Voice Interaction Design Annual Meeting 2015 by Crispin Reedy
Association for Voice Interaction Design Annual Meeting 2015Association for Voice Interaction Design Annual Meeting 2015
Association for Voice Interaction Design Annual Meeting 2015
Crispin Reedy816 views
SpeechTEK University Outtakes 2014: Zero Out Strategies by Crispin Reedy
SpeechTEK University Outtakes 2014: Zero Out StrategiesSpeechTEK University Outtakes 2014: Zero Out Strategies
SpeechTEK University Outtakes 2014: Zero Out Strategies
Crispin Reedy698 views

Recently uploaded

What’s New in CloudStack 4.19 - Abhishek Kumar - ShapeBlue by
What’s New in CloudStack 4.19 - Abhishek Kumar - ShapeBlueWhat’s New in CloudStack 4.19 - Abhishek Kumar - ShapeBlue
What’s New in CloudStack 4.19 - Abhishek Kumar - ShapeBlueShapeBlue
191 views23 slides
Declarative Kubernetes Cluster Deployment with Cloudstack and Cluster API - O... by
Declarative Kubernetes Cluster Deployment with Cloudstack and Cluster API - O...Declarative Kubernetes Cluster Deployment with Cloudstack and Cluster API - O...
Declarative Kubernetes Cluster Deployment with Cloudstack and Cluster API - O...ShapeBlue
59 views13 slides
Zero to Cloud Hero: Crafting a Private Cloud from Scratch with XCP-ng, Xen Or... by
Zero to Cloud Hero: Crafting a Private Cloud from Scratch with XCP-ng, Xen Or...Zero to Cloud Hero: Crafting a Private Cloud from Scratch with XCP-ng, Xen Or...
Zero to Cloud Hero: Crafting a Private Cloud from Scratch with XCP-ng, Xen Or...ShapeBlue
128 views20 slides
State of the Union - Rohit Yadav - Apache CloudStack by
State of the Union - Rohit Yadav - Apache CloudStackState of the Union - Rohit Yadav - Apache CloudStack
State of the Union - Rohit Yadav - Apache CloudStackShapeBlue
218 views53 slides
CloudStack Object Storage - An Introduction - Vladimir Petrov - ShapeBlue by
CloudStack Object Storage - An Introduction - Vladimir Petrov - ShapeBlueCloudStack Object Storage - An Introduction - Vladimir Petrov - ShapeBlue
CloudStack Object Storage - An Introduction - Vladimir Petrov - ShapeBlueShapeBlue
63 views15 slides
VNF Integration and Support in CloudStack - Wei Zhou - ShapeBlue by
VNF Integration and Support in CloudStack - Wei Zhou - ShapeBlueVNF Integration and Support in CloudStack - Wei Zhou - ShapeBlue
VNF Integration and Support in CloudStack - Wei Zhou - ShapeBlueShapeBlue
134 views54 slides

Recently uploaded(20)

What’s New in CloudStack 4.19 - Abhishek Kumar - ShapeBlue by ShapeBlue
What’s New in CloudStack 4.19 - Abhishek Kumar - ShapeBlueWhat’s New in CloudStack 4.19 - Abhishek Kumar - ShapeBlue
What’s New in CloudStack 4.19 - Abhishek Kumar - ShapeBlue
ShapeBlue191 views
Declarative Kubernetes Cluster Deployment with Cloudstack and Cluster API - O... by ShapeBlue
Declarative Kubernetes Cluster Deployment with Cloudstack and Cluster API - O...Declarative Kubernetes Cluster Deployment with Cloudstack and Cluster API - O...
Declarative Kubernetes Cluster Deployment with Cloudstack and Cluster API - O...
ShapeBlue59 views
Zero to Cloud Hero: Crafting a Private Cloud from Scratch with XCP-ng, Xen Or... by ShapeBlue
Zero to Cloud Hero: Crafting a Private Cloud from Scratch with XCP-ng, Xen Or...Zero to Cloud Hero: Crafting a Private Cloud from Scratch with XCP-ng, Xen Or...
Zero to Cloud Hero: Crafting a Private Cloud from Scratch with XCP-ng, Xen Or...
ShapeBlue128 views
State of the Union - Rohit Yadav - Apache CloudStack by ShapeBlue
State of the Union - Rohit Yadav - Apache CloudStackState of the Union - Rohit Yadav - Apache CloudStack
State of the Union - Rohit Yadav - Apache CloudStack
ShapeBlue218 views
CloudStack Object Storage - An Introduction - Vladimir Petrov - ShapeBlue by ShapeBlue
CloudStack Object Storage - An Introduction - Vladimir Petrov - ShapeBlueCloudStack Object Storage - An Introduction - Vladimir Petrov - ShapeBlue
CloudStack Object Storage - An Introduction - Vladimir Petrov - ShapeBlue
ShapeBlue63 views
VNF Integration and Support in CloudStack - Wei Zhou - ShapeBlue by ShapeBlue
VNF Integration and Support in CloudStack - Wei Zhou - ShapeBlueVNF Integration and Support in CloudStack - Wei Zhou - ShapeBlue
VNF Integration and Support in CloudStack - Wei Zhou - ShapeBlue
ShapeBlue134 views
DRBD Deep Dive - Philipp Reisner - LINBIT by ShapeBlue
DRBD Deep Dive - Philipp Reisner - LINBITDRBD Deep Dive - Philipp Reisner - LINBIT
DRBD Deep Dive - Philipp Reisner - LINBIT
ShapeBlue110 views
2FA and OAuth2 in CloudStack - Andrija Panić - ShapeBlue by ShapeBlue
2FA and OAuth2 in CloudStack - Andrija Panić - ShapeBlue2FA and OAuth2 in CloudStack - Andrija Panić - ShapeBlue
2FA and OAuth2 in CloudStack - Andrija Panić - ShapeBlue
ShapeBlue75 views
Updates on the LINSTOR Driver for CloudStack - Rene Peinthor - LINBIT by ShapeBlue
Updates on the LINSTOR Driver for CloudStack - Rene Peinthor - LINBITUpdates on the LINSTOR Driver for CloudStack - Rene Peinthor - LINBIT
Updates on the LINSTOR Driver for CloudStack - Rene Peinthor - LINBIT
ShapeBlue138 views
Digital Personal Data Protection (DPDP) Practical Approach For CISOs by Priyanka Aash
Digital Personal Data Protection (DPDP) Practical Approach For CISOsDigital Personal Data Protection (DPDP) Practical Approach For CISOs
Digital Personal Data Protection (DPDP) Practical Approach For CISOs
Priyanka Aash103 views
Elevating Privacy and Security in CloudStack - Boris Stoyanov - ShapeBlue by ShapeBlue
Elevating Privacy and Security in CloudStack - Boris Stoyanov - ShapeBlueElevating Privacy and Security in CloudStack - Boris Stoyanov - ShapeBlue
Elevating Privacy and Security in CloudStack - Boris Stoyanov - ShapeBlue
ShapeBlue149 views
CloudStack and GitOps at Enterprise Scale - Alex Dometrius, Rene Glover - AT&T by ShapeBlue
CloudStack and GitOps at Enterprise Scale - Alex Dometrius, Rene Glover - AT&TCloudStack and GitOps at Enterprise Scale - Alex Dometrius, Rene Glover - AT&T
CloudStack and GitOps at Enterprise Scale - Alex Dometrius, Rene Glover - AT&T
ShapeBlue81 views
"Surviving highload with Node.js", Andrii Shumada by Fwdays
"Surviving highload with Node.js", Andrii Shumada "Surviving highload with Node.js", Andrii Shumada
"Surviving highload with Node.js", Andrii Shumada
Fwdays49 views
Setting Up Your First CloudStack Environment with Beginners Challenges - MD R... by ShapeBlue
Setting Up Your First CloudStack Environment with Beginners Challenges - MD R...Setting Up Your First CloudStack Environment with Beginners Challenges - MD R...
Setting Up Your First CloudStack Environment with Beginners Challenges - MD R...
ShapeBlue105 views
Business Analyst Series 2023 - Week 4 Session 7 by DianaGray10
Business Analyst Series 2023 -  Week 4 Session 7Business Analyst Series 2023 -  Week 4 Session 7
Business Analyst Series 2023 - Week 4 Session 7
DianaGray10110 views
Future of AR - Facebook Presentation by Rob McCarty
Future of AR - Facebook PresentationFuture of AR - Facebook Presentation
Future of AR - Facebook Presentation
Rob McCarty54 views
Data Integrity for Banking and Financial Services by Precisely
Data Integrity for Banking and Financial ServicesData Integrity for Banking and Financial Services
Data Integrity for Banking and Financial Services
Precisely76 views
DRaaS using Snapshot copy and destination selection (DRaaS) - Alexandre Matti... by ShapeBlue
DRaaS using Snapshot copy and destination selection (DRaaS) - Alexandre Matti...DRaaS using Snapshot copy and destination selection (DRaaS) - Alexandre Matti...
DRaaS using Snapshot copy and destination selection (DRaaS) - Alexandre Matti...
ShapeBlue69 views

2013 Speech TEK - Alphanumeric Recognition Discussion

  • 1. © 2002 – 2012 Versay Solutions, LLC. All rights reserved. Alphanumeric Speech Recognition SpeechTek August 19, 2013 Crispin Reedy
  • 2. “The fault, dear Brutus, is not in our stars, but in ourselves” -- Julius Caesar, Act I, scene ii 2 The ProblemWithAlphanumerics
  • 3. © 2002 – 2012 Versay Solutions, LLC. All rights reserved. The Need • Account Numbers • Policy Numbers • Spelling out names and addresses • Special cases – VIN, Canadian Postal Code • And more… 3
  • 4. © 2002 – 2012 Versay Solutions, LLC. All rights reserved. Methods for Addressing • Project Tactics • Limit the grammar – Constraint List – N-Best + Back-End Data Validation • Confirmation • Prefiller 4
  • 5. © 2002 – 2012 Versay Solutions, LLC. All rights reserved. Project Tactics • Can you avoid it? – Phone number / SSN / Zip / DOB? • Set expectations – Not always easy! • Describe the problem • What tools do you have available? – Constraints / patterns? – Back-end data source available? • Can you run a proof of concept / experiment? 5
  • 6. © 2002 – 2012 Versay Solutions, LLC. All rights reserved. Constraints and Patterns • Does the number have any known pattern that can be used to limit possible values (and thereby improve recognition) – For example: • First character is always A • First three characters are always numbers • Last characters are always C, G or T. • If the answer is “no,” consider doing your own analysis. – Even if you don’t think there is a pattern, there may be one. 6
  • 7. © 2002 – 2012 Versay Solutions, LLC. All rights reserved. Applying Constraints • Writing grammar specifically for the pattern – How complicated is it? • Applying a constraint list. – How big is it? 7
  • 8. © 2002 – 2012 Versay Solutions, LLC. All rights reserved. Using nBest + Back-End Data • Collect using an unconstrained grammar • Set your recognizer to return an nBest list. • Use a webservice / back end data dip to determine which ones are “real.” • Confirm the first “real” one on the list – Throw out the ones that are not real. • If no, confirm the second “real” one on the list. – Potentially collect again after that. 8
  • 9. © 2002 – 2012 Versay Solutions, LLC. All rights reserved. Confirmation Strategy • PROTIP: Phonemes that are difficult for the recognizer to hear … are also difficult for humans to hear when they are spoken back. • Confirm using letter names for easily confusable alphanumerics. – “You said 8, 2, 7 G as in George, B as in Boy, 9. Is that right?” 9
  • 10. © 2002 – 2012 Versay Solutions, LLC. All rights reserved. What About Letter Names? • Yes with caveats: – Do you have a special domain that would allow you to teach the caller letter names? – Letter names invented by the caller will be quite variable. • Some of the “oddballs” will never be recognized – If letter names are used during confirmation, and the utterance is re-collected, the caller may tend to use those letter names during the second collection. • So add them. 10
  • 11. © 2002 – 2012 Versay Solutions, LLC. All rights reserved. What About Letter Names? • Yes, because: – Longer utterances “B as in Boy” are not likely to generate false acceptance between shorter utterances such as “G” “T” etc. • Make them separate rules so they can be weighted 11
  • 12. © 2002 – 2012 Versay Solutions, LLC. All rights reserved. Using Prefiller • “The account number is… B Z 3 9 0” – Noticeable improvement in recognition of first letter – Caller may spontaneously offer – Consider teaching the caller to say the prefiller • Especially if you have repeat callers 12
  • 13. © 2002 – 2012 Versay Solutions, LLC. All rights reserved. Other Suggestions • Look at speech recognition parameters that are not directly related to alphanumeric – Are callers calling from a very noisy environment? • Adjust overall speech threshold – Timing of utterance collection? • Listen to recording of utterances to make sure everything is getting collected 13
  • 14. © 2002 – 2012 Versay Solutions, LLC. All rights reserved. Specific Cases • VIN – Has specific pattern, but different for each manufacturer – 16 digits: nobody will want to re-enter if you get it wrong. 14
  • 15. IT DEPENDS! 15 but which way is “the best?”

Editor's Notes

  1. Or, in other words, the problem, dear friends, is not with our recognizers, but with our language. Specifically, with our English letter names, which are very confusable, i.e. E, D, T, 3, G, P, etc. etc. which all consist of a very short initial phoneme which is easily missed by recognizers and a longer phoneme which is quite confusable.