Meet Magento Poland 2018 presentation on "Integrating voice search in Magento 2 using Web speech API" by Adrian Bece from Inchoo.

1. #MM18PL#MM18PL
Integrating voice search in
Magento 2 using Web
Speech API
Adrian Bece | @AdrianBDesigns

2. #MM18PL#MM18PL
Adrian Bece
● Working at Inchoo since 2015
● React Developer
● Certified Magento 1 Frontend Developer
● Worked on various Magento 1 and Magento 2 projects for international clients
● Worked on numerous personal React projects
@AdrianBDesigns | codeadrian.github.io

3. #MM18PL#MM18PL
“Voice commerce represents the next major disruption in
the retail industry (…) The speed with which consumers
are adopting smart speakers will translate into a number
of opportunities and even more challenges for traditional
retailers and consumer products companies.”
- John Franklin, OC&C (December 2017)

4. #MM18PL#MM18PL
Voice Shopping & Home assistants
● People are getting more accustomed to Voice Shopping (Smart Speaker / Home
Assistants)
● Amazon dominates the Home assistant market with Echo device (68%)
● In late 2017, Walmart has teamed with Google to offer Voice Shopping
● Most commonly shopped categories through voice:
Groceries (20%), Entertainment (19%), Electronics (17%), Clothing (8%) *
● Voice purchases tend to be stand-alone, lower value items *
● Prediction: Voice Shopping Market in US to grow from $2 billion to $40 billion in 2022 *
* OC&C Strategy Consultants Study (December 2017)

5. #MM18PL#MM18PL
Voice Shopping & Home assistants - concerns
● Not convenient for obtaining information about product (attributes, description, price, etc.)
● Product keywords might change → voice search results might change
● 2% of the people who use Amazon’s Echo have made a purchase with their voices in 2018 *
● Of the people who used voice shopping, most people didn’t use Voice shopping again *
● People who did use Voice Shopping again, used it more extensively (searching, tracking
orders, etc.) *
● Amazon tries to encourage Voice Shopping by offering discounts and better deals
● There is potential in Voice Shopping, but there is no established pattern which is familiar to
users
* Anonymous Sources (Amazon insider info) for “The Information” (August 2018)

6. #MM18PL#MM18PL
What about regular
Web browsers?

7. #MM18PL#MM18PL
Browser Voice Search Potential
● Google Voice Search queries has grown 7x since 2010
● Android - 1 of 5 searches are done by voice (US)
● Baidu - 1 of 10 searches are done by voice
● 1 of 3 users use voice recognition technology on their device daily
● Number of voice searches expected to continue growing
● Prediction: 50% of searches will be done by voice by 2020.

8. #MM18PL#MM18PL
Web Speech API
● Provides speech-input and text-to-speech functionality in a web browser
● Web apps are able to handle voice data
● Interesting new possibilities for accessibility and control mechanisms
● Features which are not typically available when using standard software
● Experimental technology
● Two main parts
● Speech Recognition API
● provides ability to recognize voice from an audio input
● Speech Synthesis API
● provides ability to read out the text content

9. #MM18PL#MM18PL
Speech Recognition API (63% global browser support)

10. #MM18PL#MM18PL
Speech Synthesis API (80% global browser support)

11. #MM18PL#MM18PL
Speech Recognition API
● It requires internet connection to work on Chrome
● HTTPS only - can be avoided if specific flag is set when running Chrome
● It requires a config flag to be set in Firefox
● Properties
● Grammar, Language, Single / Multiple Results, Final / Interim Results
● Events
● Audio start, Audio end, On error, On result, On speech start, On speech end, etc.
● Methods
● Starting speech recognition, Stopping with returning a result, Stopping without returning a
result

12. #MM18PL#MM18PL
Speech
Recognition Data
• Settings
• Interim Results
• Single Result
• Returns
• Confidence
• Transcript

13. #MM18PL#MM18PL
Voice recognition can be used for dictating text in a form field,
as well as navigating to and activating links, buttons, and other
controls. (…) Some voice recognition tools allow complete
control over computer interaction, allowing users to scroll the
screen, copy and paste text, activate menus, and perform other
functions.
- W3C Web Accessibility Initiative on “Voice Recognition”

14. #MM18PL#MM18PL
Accessibility
● Who benefits from the Speech Recognition functionality?
● People with physical disabilities who cannot use the keyboard or mouse.
● People with chronic conditions who need to limit or avoid using the keyboard or
mouse.
● People with cognitive and learning disabilities who need to use voice rather than to
type.
● People with temporary limitations, such as a broken arm.
● People who prefer to speak rather than type (Depends on the device type,
situation, personal preference, etc.)

15. #MM18PL#MM18PL
web/js/
voiceSearch.js
• General Structure
• Options
• Setup
• Functions
• Event Listeners
• Support Functions
define([
'jquery',
'mage/translate',
'domReady!'
], function ($) {
'use strict';
$.fn.voiceSearch = function (options) {
/***********
* OPTIONS *
***********/
/*********
* SETUP *
*********/
/*************
* FUNCTIONS *
*************/
};
return function (config, node) {
$(node).voiceSearch(config);
}
});

16. #MM18PL#MM18PL
web/js/
voiceSearch.js
• Options Section
var defaults = {
"placeholderText": "Listening...",
"speechRecognitionTrigger": "#voice-search-
trigger",
"formInput": "#search"
},
$formElement = $(this);
options = $.extend(defaults, options);
$formElement = $(this);
var recognition = null;
var isActive = false;
var speechRecognitionTrigger =
$(options.speechRecognitionTrigger),
$formInput = $(options.formInput),
placeholderText =
$.mage.__(options.placeholderText);
window.SpeechRecognition = window.SpeechRecognition
|| window.webkitSpeechRecognition;

17. #MM18PL#MM18PL
web/js/
voiceSearch.js
• Setup Section
if (window.SpeechRecognition) {
recognition = new
SpeechRecognition();
recognition.interimResults = true;
this.setupRecognitionListeners();
this.setupVoiceSearchButton();
recognition.onresult =
this.handleOnResult.bind(this);
} else {
speechRecognitionTrigger.hide();
}

18. #MM18PL#MM18PL
web/js/
voiceSearch.js
• Functions Section
• Event Listeners
this.setupVoiceSearchButton = function () {
speechRecognitionTrigger.on('click touch',
this.startListening);
};
this.setupRecognitionListeners = function () {
recognition.onstart = function () {
isActive = true;
};
recognition.onend = function () {
isActive = false;
};
recognition.onerror = function () {
isActive = false;
};
};

19. #MM18PL#MM18PL
web/js/
voiceSearch.js
• Functions Section
• Support Functions
this.parseTranscript = function (e) {
return Array.from(e.results).map(
function (result) {
return result[0]
}).map(
function (result) {
return result.transcript
}).join('');
};
this.startListening = function (e) {
e.preventDefault();
if(!isActive) {
$formInput.val('');
$formInput.attr("placeholder", placeholderText);
recognition.start();
}
};
this.handleOnResult = function (e) {
$formInput.val(this.parseTranscript(e).toLowerCase());
if (e.results[0].isFinal) {
$formElement.submit();
}
};

20. #MM18PL#MM18PL
requirejs-config.js
• Standard mapping
var config = {
map: {
'*': {
'voiceSearch': 'js/voiceSearch'
}
}
};

21. #MM18PL#MM18PL
Magento_Search/templat
es/form.mini.phtml
• Add a call to the <form> element
• Add voice search <button>
• Add some id-s if necessary
• Call the voiceSearch
• Default parameters
• Custom parameters
(…)
<form
class="form minisearch"
id="search_mini_form"
action="<?= /* @escapeNotVerified */
$helper->getResultUrl() ?>"
method="get"
data-mage-init=‘{"voiceSearch":{}}'>
(…)
<button class="action search voice"
id=“voice-search-trigger"></button>
(…)
(…)

22. #MM18PL#MM18PL

23. #MM18PL#MM18PL
Result -
Page Load
• If Web Speech API is not supported, Voice search
button will be hidden
• User will be prompted to confirm microphone
permission
• After user grands permission, the text input
placeholder will change to “Listening…”

24. #MM18PL#MM18PL
Result - Search
Action
• After user has finished speaking, text will be added
to the search input form
• Form submits and user is redirected to Magento 2
search results page
• User can click again on the Voice Search button
and perform search from the beginning

25. #MM18PL#MM18PL
Areas Of Improvement
● Confirm the input before submitting the form
● Cancel voice search if input is not received in some time limit
● Check if the confidence of the result is above certain percentage before submitting.
If percentage is lower than the threshold, ask user to repeat the voice search.
● UX improvement - Better communication of current state (recording, submitting
form, etc.)
● Open the product page for the first search result instead of search result page
● Use Google Cloud Speech-to-Text (paid) to enable language recognition and allow
users to search in their native language.

26. #MM18PL#MM18PL
Additional Ideas
● Voice-controlled shopping
● Search, selecting products
● Selecting quantity, configuring, adding to cart
● Navigating to cart and checkout
● Voice-controlled On-screen assistant
● Speech Synthesis API
● Improved accessibility, voice-controlled shortcuts
● Voice-controlled shopping, contact forms, chat support, shortcuts, etc.

27. #MM18PL#MM18PL
There might be very few voice shoppers right
now… but once you find out what’s special
about them, you can grow them very quickly.
That’s an unresolved problem.
- The Information, “The Reality Behind Voice Shopping Hype”
(August 2018)

28. #MM18PL#MM18PL
Conclusion
● People are still getting accustomed to Voice Shopping (home assistants and smart speakers)
● Voice search continues to grow across various desktop and mobile devices
● Web Speech API opens the door for speech recognition from a Web Browser (it’s still
experimental)
● Beneficial for accessibility, also has potential for regular use
● Consider adding the functionality in stages and monitor how customers are using it
● Are they using it for product discovery or shop for familiar items?
● Are they using it more on mobile because it’s easier?
● Which voice-controlled functionalities are they using more and which they are using
less?

29. #MM18PL#MM18PL
Thank you for your
attention.
Adrian Bece | @AdrianBDesigns