Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Using the Power of Speech in Your Windows Phone Apps

804 views

Published on

The Windows Phone SDK provides APIs which make it easy for developers to incorporate speech recognition and voice commands within a Windows Phone app. In this session, you will learn how to incorporate speech synthesis, voice commands, and simple speech recognition within your own Windows Phone app to provide a truly engaging user experience.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Using the Power of Speech in Your Windows Phone Apps

  1. 1. Using the Power of Speech in your Windows Phone Apps Lori Lalonde Twitter: @loriblalonde Blog: geekswithblogs.net/lorilalonde
  2. 2. About Me Twitter: @loriblalonde Email: loriblalonde@gmail.com Blog: geekswithblogs.net/lorilalonde LinkedIn: http://ca.linkedin.com/in/lorilalonde CTTDNUG
  3. 3. Why should I integrate speech in my mobile app?
  4. 4. Minimum Requirements Windows Phone 8.1 Development Windows 8.1 (x64) Visual Studio 2013 Update 2 or later (Pro, Premium, or Ultimate) OR Visual Studio 2013 Express for Windows Update 2 or later *includes Windows Phone 8.1 SDK *Windows Phone Emulators – require Professional edition of Windows, and a processor that supports Client Hyper-V and Second Level Address Translation (SLAT)
  5. 5. Speech for Windows Phone Voice Commands Speech Recognition Text-To-Speech
  6. 6. Voice Commands Simply say something to launch (or navigate into) the app… … instead of: tap, tap, scroll, select, dang… go back… tap, scroll… oh geez forget it!
  7. 7. Add Capabilities
  8. 8. Add a Voice Command Definition File <?xml version="1.0" encoding="utf-8"?> <VoiceCommands xmlns="http://schemas.microsoft.com/voicecommands/1.0"> <CommandSet xml:lang="en-US"> <CommandPrefix>Contoso Rodeo</CommandPrefix> <Example> play a new game </Example> <Command Name="PlayGame"> <Example> play a new game </Example> <ListenFor> [and] play [a] new game </ListenFor> <ListenFor> [and] start [a] new game </ListenFor> <Feedback> Starting a new game... </Feedback> <Navigate /> </Command> ... </CommandSet> </VoiceCommands>
  9. 9. Phrase Lists <Command Name="PlayLevel"> <Example> replay level two </Example> <ListenFor> replay level {number} </ListenFor> <Feedback> Going to level {number}... </Feedback> <Navigate /> </Command> <PhraseList Label="number"> <Item> one </Item> <Item> two </Item> <Item> three </Item> </PhraseList>
  10. 10. Register the VCD using Windows.Media.SpeechRecognition; using Windows.Storage; ... private async void InitializeVoiceCommands() { Uri vcdUri = new Uri("ms-appx:///MyVoiceCommands.xml", UriKind.Absolute); StorageFile file = await StorageFile.GetFileFromApplicationUriAsync(vcdUri); await VoiceCommandManager.InstallCommandSetsFromStorageFileAsync(file); } Installs the CommandSet whose specified language matches the device language setting
  11. 11. Handle Navigation protected override void OnActivated(IActivatedEventArgs args) { if (args.Kind == ActivationKind.VoiceCommand) { Frame rootFrame = Window.Current.Content as Frame; VoiceCommandActivatedEventArgs vcArgs = (VoiceCommandActivatedEventArgs)args; //check for the command name that launched the app string voiceCommandName = vcArgs.Result.RulePath.FirstOrDefault(); if (voiceCommandName == "ViewEntry") { rootFrame.Navigate(typeof(ViewDiaryEntry), vcArgs.Result.Text); } } }
  12. 12. DEMO!!!
  13. 13. Speech Synthesis (Text-To-Speech)  Enable the app to read text to the user  The text can be:  A simple string  Speech Synthesis Markup Language (SSML) string  SSML file
  14. 14. Speak! Part 1 using Windows.Media.SpeechSynthesis; ... private SpeechSynthesizer synthesizer; public async void SpeakAsync(string textToSpeech) { SpeechSynthesizer synthesizer = new SpeechSynthesizer(); SpeechSynthesisStream synthesisStream = await synthesizer.SynthesizeTextToStreamAsync(textToSpeech); ... }
  15. 15. Speak! Part 2 public async void SpeakAsync(string textToSpeech) { SpeechSynthesizer synthesizer = new SpeechSynthesizer(); SpeechSynthesisStream synthesisStream = await synthesizer.SynthesizeTextToStreamAsync(textToSpeech); if (synthesisStream != null) { this.media.AutoPlay = true; this.media.SetSource(synthesisStream, synthesisStream.ContentType); this.media.Play(); } } <MediaElement x:Name="media“ AutoPlay="False" /> Add MediaElement to the view’s XAML Set the media source to the stream and play
  16. 16. DEMO!!!
  17. 17. Enable the app to react when the user is speaking to it Speech Recognition
  18. 18. Built-In Speech Prompts/Confirmations Heard You Say… Unrecognized input Did You Say… Listening…
  19. 19. using Windows.Media.SpeechRecognition; ... private SpeechRecognizer speechRecognizer; public async Task InitializeSpeechRecognizerAsync(string topicHint) { speechRecognizer = new SpeechRecognizer(); //add web search grammar to the recognizer SpeechRecognitionTopicConstraint topicConstraint = new SpeechRecognitionTopicConstraint(SpeechRecognitionScenario.Dictation, topicHint); speechRecognizer.Constraints.Add(topicConstraint); await speechRecognizer.CompileConstraintsAsync(); } Initialize SpeechRecognizer A speech recognizer needs at least one constraint, and constraints must be compiled, before speech recognition can be performed.
  20. 20. private SpeechRecognizer speechRecognizer; public async void SpeakTextAsync() { SpeechRecognitionResult recognitionResult = await speechRecognizer.RecognizeWithUIAsync(); ... } Using the Built-In UI
  21. 21. private SpeechRecognizer speechRecognizer; public async void SpeakTextAsync(string audiblePrompt, string exampleText) { speechRecognizerUI.UIOptions.IsReadBackEnabled = true; speechRecognizerUI.UIOptions.ShowConfirmation = true; speechRecognizer.UIOptions.AudiblePrompt = audiblePrompt; speechRecognizer.UIOptions.ExampleText = exampleText; SpeechRecognitionResult recognitionResult = await speechRecognizer.RecognizeWithUIAsync(); ... } Setting UI Options
  22. 22. DEMO!!!
  23. 23. Questions
  24. 24. Additional Resources MSDN - Speech Design Guidelines for Windows Phone Apps MSDN - Quickstart: Speech Recognition (XAML) MSDN - Responding to speech interactions (XAML) Channel9 Video: Integrating Your App into the Windows Phone Speech Experience W3C Speech Recognition Grammar Specification W3C Speech Synthesis Markup Language
  25. 25. apress.com 30% off Windows Phone 8 Recipes eBook Discount Code: LORI14 Expires: Dec 31, 2014
  26. 26. Lori Lalonde Twitter: @loriblalonde Blog: geekswithblogs.net/lorilalonde

×