Speech recognition: ready to take off?

By: Ma Jie (A0129447X)
Niu Rui (A0040287J)
Nguyen Gia Huy (A0045581E)
Liu Lili (A0132407R)
Tan Gee Kwang (A0147159X)
Speech Recognition:
Ready to Take Off?

Overview
• Siri
• Other applications
Performance
of SR
• Underlying technologySR
improvement
• Avionics
• Field Automation
Emerging
Application

 In 2013, Intelligent Voice survey showed
that only 15% of respondents said that
they had used Siri in iOS7. Nearly half
believed Apple had “oversold Siri’s voice
recognition capabilities”
 2015 WWDC, Apple’s software
engineering vice president claimed that
Siri Gets 1 Billion Requests a Week
Performance of Siri
 Doing Basic Math faster
 Find facts two times faster
 Four Time faster than you to set alarms
 Tweets more than two times faster than you
 Convert measurements

Siri Usage Rate Detail and Customer Satisfaction
Source: http://www.imore.com/siri-months-community-report-card
15%
36%
10%
20%
12%
7%
Do you use Siri on your iOS device?
Yes, and I like it
Yes, but it could be better
Yes, and I'm neutral
No: tried it and didn't like it
No: I didn't even try because I have no desire
Other
Source: http://www.besttechie.com/2013/03/07/do-people-still-use-siri/

Performance of Siri
 Apple claims that iOS 9, Siri
will be up to 40 percent faster
and 40 percent more accurate
What has hold it back?
1. There is learning curve.
2. It’s far from perfect
3. The use cases are limited
4. Lack of integration of third-party apps

Speech Recognition Market
Source: Matt M., Joshua S., and David H. 2014. Dynamic Commercialization Strategies for Disruptive Technologies: Evidence from the Speech Recognition Industry
 In past 50 years, the technological
breakthroughs haven enabled the SR
become reality.
 Coupled with the advances in CPU
power and enhanced software
algorithms, SR had achieved steep
improvement and commercial
feasibility after 1990s.

Current Applications of SR
Applications in various industries
 Call Centers
 Medical Industries
 Education
 Automotive
 Home Automation

 Students with disabilities used a SR powered Hosted
Transcription System (HTS) to convert digitized audio
and video into accessible, Multimedia Transcripts
 In 2011, 52% of Canadian disability service providers
interviewed reported using speech to text supports
 Strengthen by lowering WER
 Problems:
– Scalability to meet temporal demands
– Fixed cost for infrastructure
SR in Educational – Liberated learning project (LLR)
Quality
Cost
Source: http://www.transcribeyourclass.ca/financial.html

 HIS Automotive: About 25% U.S. motorists use speech recognition in their cars daily
and 53% use it at least once a week; by 2020, 68 million vehicles worldwide will
have voice controls, increased by 84% from 37 million in 2014.
SR in Automotive

 Most SR in today’s market have about 50 to 60 voice commands
 Common used features: Make calls, play music, temperature control,
navigation.
 More features available: Reminders, Send emails, search nearby
restaurants/shops/petrol stations, real-time traffic conditions,
connect to other SR control system (e.g. home automation)…
SR in Automotive

 Nuance – Dragon Drive Platform
– Cloud-based voice and content solutions
– Integrated with in-vehicle cloud-based search capabilities from Telenav, leader of
location-based services (Source: Telenav, Nov 3, 2015)
– Attractive features – Read out the daily update when enters the car, Connect the
home to your car through LG HomeChat software
SR in Automotive
Video: https://www.youtube.com/watch?v=laxXWUxXcWs

 Problems encountered with ASR in cars -
– Doesn’t recognize/misinterprets verbal commands (63 percent)
– Doesn’t recognize/misinterprets names/words (44 percent)
– Doesn’t recognize/misinterprets numbers (31 percent)
– Wind noise
– Language accents
– Imperfect speech recognition software might prove to be a distraction
SR in Automotive

SR in Home Automation
 Smart home
– Lighting control (Vocca)
– TV (apple TV)
– Personal Assistant (Echo, Homey)

SR in Home Automation – Apple TV
The Apple TV uses Siri search as the glue that holds all
those individual apps together. Voice commands (also
found on Roku, Android TV and Amazon Fire TV) are
easier than entering names on a virtual keyboard. And
despite some rough edges, Siri is more helpful than
the rest.
Siri’s advantage is more advanced queries.
 Six degrees of Kevin Bacon
 Filter TV episodes by actors
 Rewind
Siri’s limitation:
 Pronunciation of difficult names
 TV show recognition by genres
Source: http://www.wsj.com/articles/apple-tv-review-a-giant-iphone-for-your-living-room-1446080460
The TV of the future needs to be as powerful and easy
to use as an iPhone, and this Apple TV is the first
box—and the first Apple TV—to achieve that.

 Amazon Echo – launch in
November 6, 2014 Limited
and June 23, 2015 Wide
 Can answer general
questions, reorder the
items you buy frequently
from Amazon, and play
music
Source: http://www.amazon.com/Amazon-SK705DI-
Echo/dp/B00X4WHP5E/ref=sr_1_1?ie=UTF8&qid=14
46173814&sr=8-1&keywords=amazon+echo
Source: http://www.cnet.com/products/amazon-echo-review/

 Apple's HomeKit
– A framework for communicating with and controlling connected accessories
in a user’s home, announced in Apple WWDC 2014.
HomeKit-certified devices
ecobee3 Use sensors and a thermostat to keep tabs on your home’s temp.
Elgato
A variety of Elgato’s Eve sensors will give you all kinds of information about what’s going on inside your home.
(Door & Window, Energy, Weather, Room)
iHome Connect ordinary devices into the smart plug, and you can start controlling them with your phone.
Insteon The company’s hub can control all its products, including lights and locks, even from outside your home.
Lutron Control your lights and shades with its bridges and kits.
iDevices
Plug anything into the company’s indoor or outdoor switch to make the device smart, and control your climate
with the thermostat.
Schlage You’ll be able to ask Siri to lock and unlock your door.
August
The smart lock company announced a doorbell camera and keypad to its lineup, but it’s just the new lock that
works with Siri for now.
Coming Plugs, Thermostats (Honeywell Lyric), Lighting (Philips), Alarm System (Honeywell Lynx Security System)
Partnerships
Chamberlain MA Garage, Cree, Friday Smart Lock, GE (color-changing LEDs), Haier (smart air-conditioner),
Incipio, Kwikset, Netatmo, Osram Sylvania, Philips Hue, SkyBell, Withings (baby monitors)
Source: http://www.digitaltrends.com/home/a-list-of-apple-homekit-compatible-devices/
Total price:
US$2000

Source: http://publications.lib.chalmers.se/records/fulltext/203117/203117.pdf
Most common
used features
Other features that
users would like

 There is user base for SR (doctors, drivers, smart phone users…)
 But the fact is that most of the customers only tried few times or use basic
commands for SR when they have to (driving, busy hands, etc.)
 Why?
– SR doesn’t recognize the complicated commands, which offers limitations to the
features
– SR reacts very slow
– Takes time to train it
– Interaction with SR is not natural; words must be clear and without emotion
– Bad first impression, no interest to try even SR is improving
Summary of Challenges in SR
Customers don’t think that using SR is necessary in their daily life!

ComponentsRequirementsDimension
Speed
Process the
algorithms
Processor
Underlying Technology of Speech Recognition
Source: http://web.sfc.keio.ac.jp/~rdv/keio/sfc/teaching/architecture/architecture-
2008/lec07-cache.html

AchievementsRequirementsDimension
Accuracy
Quality of
Signal Receive
Background
noise
elimination
Channel effect
elimination
Acoustic
scoring
Deep Learning
Acoustic
database
Language
Matching
Modelling
Language
database

Accuracy
Quality of
Signal Receive
Background
noise
elimination
Channel effect
elimination
Acoustic
scoring
Deep Learning
Acoustic
database
Language
Matching
Modelling
Language
database
Microphone
Components

Accuracy
Quality of
Signal Receive
Background
noise
elimination
Channel effect
elimination
Acoustic
scoring
Deep Learning
Acoustic
database
Language
Matching
Modelling
Language
database
Memory
Components
• Speech Recognition needs support from
data base which can be local or in Cloud.
• Performance of memory is far behind
processor, bottleneck of SRS is memory
speed (network speed if with Cloud)
Source: http://web.sfc.keio.ac.jp/~rdv/keio/sfc/teaching/architecture/architecture-
2008/lec07-cache.html

Accuracy
Quality of
Signal Receive
Background
noise
elimination
Channel effect
elimination
Acoustic
scoring
Deep Learning
Acoustic
database
Language
Matching
Modelling
Language
database
Algorithms
Components

Noise Elimination Algorithm Performance
• Noise has two main effects over the speech representation: distortion in the
representation space, and a loss of information.
• Study shows that noise compensation methods will help to improve the accuracy in
different SNR (signal noise ratio) levels and distances
Source: Angel de la T. et al. Speech Recognition Under Noise
Conditions: Compensation Methods
Source: Pedro J. Moreno, 1996, Speech Recognition in Noisy Environments

 Speakers may have different accents, dialects, or pronunciations, and
speak in different styles, at different rates, and in different emotional
states.
 Deep learning, introduced in 2006, attempt to learn multiple levels of
representation of increasing complexity/abstraction.
 A new architecture, the deep belief network (DBN)-HMM, has been
developed in 2012.
Deep Learning

 Idea was started from 1970s, but the
progress is very slow -> Computational
and data limitations
 Deep learning - one step closer to
artificial intelligence
Deep Learning
More data Faster hardware

 Word error rate (WER) for SR technology in automotive has been reduced to
below <1%
Accuracy of SR
Source: http://whatsnext.nuance.com/in-the-labs/deep-learning-in-connected-cars/

 Overall WER improvement for SR
Accuracy of SR
Source: http://whatsnext.nuance.com/in-the-labs/what-is-deep-machine-learning/

Accuracy of SR
 According to Baidu, their error rates in a clean environment were at 6.56% and
19.06% in noisy environments by using GPUs
 Apple claims that Siri in iOS 9 has only a 5% word error rate
 Siri in iOS 9 requests to teach Siri your voice whenever change to a new language
Source: NVIDIA GTC: The Race To
Perfect Voice Recognition Using GPUs
TARGET: < 0.1% or even 0%

How will SR improve further?
Customers don’t think that using SR is necessaryin their daily life!
BUT IF –
SR is faster and smarter to understand the commands, with more
features available
Customers might start thinking: Why not try SR?
For example: Ability to recognize multilingual content, direct link to
third-party apps, allow multi-users to interact at the same time…

 So, when will SR like Siri be able to widely used by customers?
 2020 to 2025
– Improvement of Deep Learning (Apple has just acquired VocalIQ in Oct, 2015) for
more intelligent algorithm
– Improvement of Big data, multiple channels to enhance data base used in
modeling for higher accuracy
– Improvement of Mobile network, faster response for better customer experience
– With diffusion of smart devices and apps, new customers will get more chance to
accept SR before old hobby formed
– Potential new standard of human-machine interface
– Cost will be reduced further with core components improvement
How will SR improve further?

Speech Recognition: Future Market Trend
 Voice will be the most important area for growth in mobile user interfaces
 Tractica forecasts the growth rate for SR: reach $5.1 billion by 2024 at a CAGR of 40%
 Strongest market - Consumer-facing market: Mobile device authentication and
control of wearable devices

Global Automotive Voice Recognition Market 2014-
2018 forecasts the automotive voice recognition
sector to grow at 10.59% CAGR to 2018
 SR market in Automotive

 Market for Home automation
– Annual growth rate can reach 67% over next 5 years
– Revenue arrives $61billion with 52% compound
annual growth rate, forecast the value can reach
$490 billion in 2019

SR in Avionics - Head-in and Head-out in cockpit
 Multi-function displays with menu
structures many tiers deep
 Pilot needs one hand on collective
while the other one on the joystick

SR in Avionics
 Speech recognition reduce
workload and free hands for pilots.
 With increment of head up time,
pilot can focus on flying the aircraft
and response to out environment.
 Noise elimination and integration
with onboard system
http://www.speech.sri.com/press/airforce-
print-news-oct15-2007.pdf http://www.gizmag.com/go/7484/

Navigation Functions
• Entering waypoints and inputting FMS data
• Reduce confusion
Communication Functions
• Change frequencies of channel by voice control
• Query system by “asking”
Checklist
• Task list
• Avionics monitor
 Safety and security are roadblocks for SR adoption in avionics
 Entry level functions with low safety concerns
SR in Avionics

SR Deployment in Avionics
2000 2007 2008 2014 2015
Typhoon Gazelle F-35 & F-22 Sferion Assistance System
Direct input voice system Speaker- independent system
Start in civil avionics
Pro Line Fusion flight deck

 It is not a technology problem, but more of an acceptance problem.
 Air transport will accept after SR product actually comes out and
proves its value
SR Commercialization in Avionics
"We've hit our sweet spot finally and its gotten to the point where its getting very,
very close to being product ready in terms of being mature enough to get out
there."
- Geoff Shapiro from Rockwell Collines
Resource: http://www.aviationtoday.com/av/topstories/Rockwell-Collins-Rapidly-Advancing-Cockpit-Voice-Recognition-
Technology_83515.html#.Vjm710b0wTY

SR in Field Automation
 Equipment inspection in the field by using portable devices embedded with speech recognition
system
 Enter data faster and reduce the cost

Source: https://www.earthworksaction.org/issues/detail/oil_and_gas_noise#.Vh_SNN-qpBc
http://www.ehjournal.net/content/14/1/18
SR in Field Automation
 Noise level is very high thus noise elimination will be more challenging

 Robot designed for
dedicate functions can only
receive pre-defined
instruction
 Low request for noise
elimination, process and
memory
SR in Personal Robot for Family

Artificial Intelligence – Key technology for future improvement of SR
 We should “talk” rather
than type
 Artificial Intelligence should
be deployed in any complex
environment with capacity
to understand the
instruction
 High request for noise
elimination, process and
memory

SR in the future – everywhere in your life
Driving in the car Shopping in the mall Eating in the canteen

Speech recognition: ready to take off?

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Speech recognition: ready to take off?

Similar to Speech recognition: ready to take off? (20)

More from Jeffrey Funk

More from Jeffrey Funk (20)

Recently uploaded

Recently uploaded (20)

Speech recognition: ready to take off?