Building a New Generation of Mobile Games with Speech summarized in 3 sentences: Developers can differentiate apps and increase engagement by adding speech features which can benefit games, accessibility is important for conditions requiring hands-free use, and examples of iconic games that incorporated speech include Halo with voiced characters and Battlefield 4 with voiced squad commands.

1. Nick Landry
Senior Technical Evangelist – Microsoft
nick.landry@microsoft.com
Blog: AgeofMobility.com
@ActiveNick | github.com/ActiveNick
Building a New Generation
of Mobile Games
with Speech

4. Computer Speech is the Dream

5. Developer Opportunity
Differentiate your app with voice commands
Delight your users with natural interactions
Increase user engagement
Increase productivity
Increase discovery
Games can benefit too!

6. Mobile Speech is a Responsibility

7. Accessibility
Medical
Conditions
Hands-free
Apps &
Games
Learning &
Therapy
Travel Apps
with
Translation

8. Wing Commander
Early use of speech
Halo
Iconic voiced character
Battlefield 4
Voiced squad commands
SWTOR
Largest voice acting game

9. In-Game Voices
Voiced narrator
Talking computers
Robots & droids
Augmented Gameplay
In-game commands
Squad tactics
Speech-based Games
Text adventures
Hands-free games
Serious Games
Disabled gamers
Learning & therapy

10. Speech SDK
Features for Devs
Windows 10
Windows/WP
8.1
iOS
(iPhone/iPad)
Android
(Phone/Tablet)
Built-in Personal
Assistant with
Speech
Cortana Cortana
(WP8.1 only)
Siri Google Now
Launcher
Personal Assistant
Extensions in third-
party apps using
Voice Commands
Yes Yes
(WP8.0+ only)
No
(first-party apps
only)
No
(invite-only
third-party apps)
Speech Synthesis
SDK for Devs
(Text-to-Speech)
Yes Yes Yes Yes
In-app Speech
Recognition SDK
for Devs
Yes Yes
(WP8.0+ only)
No
(entry field
dictation only)
Yes

11. 





12. http://www.w3.org/TR/speech-synthesis/

13. <speak
version='1.0'
xmlns='http://www.w3.org/2001/10/synthesis'
xml:lang='en-us'>
<voice name='Microsoft Zira Mobile'>
<prosody pitch='low' rate='0.9'>
This is the text that will be read by the speech synthesizer.
</prosody>
</voice>
</speak>

14. Windows 10/8.1
Store Apps
Windows Phone
8.1 Apps
Android 5.0
Apps
iOS 8.1
Apps
Basic Speech Synthesis (TTS) Yes Yes Yes Yes
Male & Female Voices Yes Yes Partial* Partial*
No of Languages/Variants
Supported
16 15 16 37
Use Speech Synthesis Offline Yes Yes Yes Yes
Control TTS Voice Rate / Speed Yes Yes Yes Yes
Control TTS Voice Pitch Yes Yes Yes Yes
Control TTS Voice Volume Yes Yes Yes Yes
SSML Support Yes* Yes* Yes* No
Stop, Pause, Resume Speech Audio Yes Yes Yes* Yes
Save Synthesized Speech to Audio
File
Yes Yes Yes No
Play TTS Audio in Background Yes Yes Yes* Yes
Support for Custom TTS Engines No No Yes No
Sources: [Windows] http://msdn.microsoft.com - [Android] http://developer.android.com/reference/android/speech/tts/package-summary.html
[iOS] https://developer.apple.com/library/ios/documentation/AVFoundation/Reference/AVSpeechSynthesizer_Ref/index.html

19. http://projectoxford.ai

20. Microphone Array
http://kinectforwindows.com

22. HoloLens “X-Ray” Tech Demo / Game

23. Call
Text
Remind
Email
Voice Command
XAdventure Works
Voice Command
Definition (VCD)
Microsoft Speech
Platform Service (cloud)
Adventure Works, show
my trip to London
Application Code

25. Call to Action: Speech-enable your games!
• Speech is a convenience, it’s fun, and it’s also a responsibility!
• Speech synthesis is the low-hanging fruit
• Give users quick and easy access to frequently
accessed game launching scenarios
via Voice Commands
• Can you think of games that can be
played completely hands-free?
• Think outside the box!

26. https://github.com/activenick
http://aka.ms/cortanasample
http://aka.ms/grouppost
http://aka.ms/trailfinder81
http://aka.ms/cortanadocs
http://aka.ms/speechdocs
https://msdn.microsoft.com/en-us/library/windows/apps/dn974233.aspx
http://dev.bing.com
https://github.com/Microsoft/unityplugins

27. BUILD 2015 Sessions On-Demand
Cortana and Speech Platform In-depth Adina Trufinescu
Khuram Shahid
Session Title Speakers Session Code
3-716
Build with Bing: Use Bing APIs and
Tools to Enrich Your Apps and Make
Them More Discoverable
Andre Furtado
Vincent Wehren
2-727
Hosted Web Apps and Web
Platform Innovations
Jeff Burtoft
Kiril Seksenov
2-665
Using Azure Search to Build Great
Search Experiences in Mobile and
Web Apps
Pablo Castro 2-745
Cortana Extensibility in Universal
Windows Apps
Ed Doran
Ali Emami
2-691
http://channel9.msdn.com/events/build/2015

28. https://github.com/Microsoft/Windows-universal-samples
http://channel9.msdn.com/events/build/2015
http://aka.ms/win10devmva
http://aka.ms/devassistant
http://blogs.windows.com/buildingapps/2015/08/25/using-cortana-to-
interact-with-your-customers-10-by-10

29. Online Microsoft training delivered by experts
to help technologists continually learn
Hundreds of courses for developers, IT Pros,
students, entrepreneurs and enthusiasts
11 different languages
3M+ students registered
Build your own Learning Plan
All free!
http://www.microsoftvirtualacademy.com

30. • Universal Windows App Development
with Cortana and the Speech SDK
• Available for on-demand viewing now:
http://aka.ms/CortanaMVA

31. • Channel 9 Show
• Visual Studio Toolbox
with Robert Green
https://channel9.msdn.com/Shows/Visual-Studio-Toolbox/App-Development-with-Cortana

32. Visual Studio
Community Edition
Modern Web
Development
Mobile App
Development
Cloud
Development
Game
Development

33. • Apply at aka.ms/bizsparkapply

34. • Visual Studio Enterprise Edition
• Office 365 Subscription
• Windows licenses
• Server software licenses
• All licenses are yours to keep
• No commitment to buy
aka.ms/bizsparkapply

35. Slides are in SlideShare. Demos are on GitHub.
Slideshare: www.slideshare.net/ActiveNick
Blog: www.AgeofMobility.com
Twitter: @ActiveNick
Mobile Apps: www.bigbaldapps.com
LinkedIn: www.linkedin.com/in/activenick
GitHub: github.com/ActiveNick
Email: nick.landry@microsoft.com