Nick Landry
Senior Technical Evangelist – Microsoft
nick.landry@microsoft.com
Blog: AgeofMobility.com
@ActiveNick | github.com/ActiveNick
Microsoft Cognitive Services:
Building Smart Applications
with Speech, NLP & Vision
Microsoft
Cognitive
Services
Give your apps
a human side
Microsoft
Cognitive
Services
Give your apps
a human side
Cognitive Services API Collection
Bringing it all together
The Seeing AI App
Computer Vision, Image, Speech Recognition, NLP,
and ML from Microsoft Cognitive Services
Watch Video HereRead Blog Here
Emotion
Speaker
Recognition
Speech
Custom
Recognition
Computer Vision
Face
Video
microsoft.com/cognitive
Linguistic Analysis
Language
Understanding
Bing Spell Check
Entity Linking
Knowledge
Exploration
Academic
Knowledge
Bing
Image Search
Bing
Video Search
Bing
Web Search
WebLM
Text Analytics Recommendations
Bing
Autosuggest
Bing
News Search
Translator
Emotion
Speaker
Recognition
Speech
Custom
Recognition
Computer Vision
Face
Video
microsoft.com/cognitive
Linguistic Analysis
Language
Understanding
Bing Spell Check
Entity Linking
Knowledge
Exploration
Academic
Knowledge
Bing
Image Search
Bing
Video Search
Bing
Web Search
WebLM
Text Analytics Recommendations
Bing
Autosuggest
Bing
News Search
Translator
Computer Vision API
Distill actionable
information from
images
Video API
Analyze, edit, and
process videos within
your app
Face API
Detect, identify,
analyze, organize, and
tag faces in photos
Emotion API
Personalize
experiences with
emotion recognition
Vision
Updated Computer Vision API
Content of Image:
Categories v0: [{ “name”: “animal”, “score”: 0.9765625 }]
V1: [{ "name": "grass", "confidence": 0.9999992847442627 },
{ "name": "outdoor", "confidence": 0.9999072551727295 },
{ "name": "cow", "confidence": 0.99954754114151 },
{ "name": "field", "confidence": 0.9976195693016052 },
{ "name": "brown", "confidence": 0.988935649394989 },
{ "name": "animal", "confidence": 0.97904372215271 },
{ "name": "standing", "confidence": 0.9632768630981445 },
{ "name": "mammal", "confidence": 0.9366017580032349,
"hint": "animal" },
{ "name": "wire", "confidence": 0.8946959376335144 },
{ "name": "green", "confidence": 0.8844101428985596 },
{ "name": "pasture", "confidence": 0.8332059383392334 },
{ "name": "bovine", "confidence": 0.5618471503257751,
"hint": "animal" },
{ "name": "grassy", "confidence": 0.48627158999443054 },
{ "name": "lush", "confidence": 0.1874018907546997 },
{ "name": "staring", "confidence": 0.165890634059906 }]
Describe
0.975 "a brown cow standing on top of a lush green field“
0.974 “a cow standing on top of a lush green field”
0.965 “a large brown cow standing on top of a lush green field”
Start coding
microsoft.com/cognitive
Speech
Bing Spell
Check API
Detect and correct
spelling mistakes
within your app
Language Understanding
Intelligent Service
Teach your apps to
understand
commands from
your users
Web Language
Model API
Leverage the power
of language models
trained on web-scale
data
Linguistic
Analysis API
Easily parse complex
text with language
analysis
Text Analytics
API
Detect sentiment,
key phrases, topics,
and language from
your text
Language
Reduce labeling effort with interactive featuring
Seamless integration to Speech API
Deploy using just a few examples with active learning
Supports 5 languages (English, Chinese, Italian, French, Spanish)
Language Understanding Models
{
“entities”: [
{
“entity”: “flight_delays”,
“type”: “Topic”
}
],
“intents”: [
{
“intent”: “FindNews”,
“score”: 0.99853384
},
{
“intent”: “None”,
“score”: 0.07289317
},
{
“intent”: “ReadNews”,
“score”: 0.0167122427
},
{
“intent”: “ShareNews”,
“score”: 1.0919299E-06
}
]
}
“News about
flight delays”
Language Understanding Models
Your conversation logic
Logic
Web
Service
Your Bot
LUIS
Start coding
microsoft.com/cognitive
https://www.microsoft.com/cognitive-services/en-us/text-analytics-api
Start coding
microsoft.com/cognitive
https://www.microsoft.com/en-us/translator/products.aspx
Start coding
microsoft.com/cognitive
Easy
Flexible
Quality
Why Microsoft
Cognitive Services?
Developer
Resources
Preview Pricing
https://www.microsoft.com/cognitive-services/en-us/pricing
Documentation
https://www.microsoft.com/cognitive-services/en-us/computer-vision-api/documentation
Client SDKs
https://github.com/Microsoft/ProjectOxford-ClientSDK
https://github.com/felixrieseberg/project-oxford (nodejs)
https://github.com/southwood/project-oxford-python
Example Code
https://github.com/jsturtevant/happy-image-tester-django
https://github.com/jsturtevant/happy-image-tester-nodejs
Join Our Community
https://stackoverflow.com/questions/tagged/microsoft-cognitive
https://social.msdn.microsoft.com/forums/azure/en-US/home?forum=mlapi
https://cognitive.uservoice.com/
https://blog.xamarin.com/performing-ocr-for-ios-android-and-windows-
with-microsoft-cognitive-services/
https://github.com/pierceboggan/smarter-apps/tree/master/ExpenseIt/
https://blog.xamarin.com/adding-facial-recognition-to-your-mobile-apps/
https://github.com/pierceboggan/smarter-
apps/tree/master/talk/employee-directory/
https://github.com/xamarin/mini-hacks/tree/master/microsoft-cognitive-
services/solutions
https://github.com/DotNetToscana/See4Me
https://github.com/ActiveNick/BingTranslateDemo
https://github.com/ActiveNick/CognitiveServices-Demo-Face
https://github.com/ActiveNick/CognitiveServices-Demo-
EntityLinking
https://github.com/ActiveNick/CognitiveServices-Demo-
TextAnalytics
https://channel9.msdn.com/Events/Build/2016/B878
https://channel9.msdn.com/Events/Build/2016/B855
https://channel9.msdn.com/Events/Build/2016/T620
https://channel9.msdn.com/Events/Build/2016/B834
https://channel9.msdn.com/Events/Build/2016/B833
http://developer.microsoft.com/cortana
http://AgeofMobility.com/Cortana
Online Microsoft training delivered by experts
to help technologists continually learn
Hundreds of courses for developers, IT Pros,
students, entrepreneurs and enthusiasts
11 different languages
3M+ students registered
Build your own Learning Plan
All free!
http://mva.microsoft.com
• Universal Windows App Development
with Cortana and the Speech SDK
• Available for on-demand viewing now:
http://aka.ms/CortanaMVA
• Channel 9 Show
• Visual Studio Toolbox
with Robert Green
• New Voice Commands
• Integration with Cortana’s canvas
• Background Voice Commands
• Continuous dictation
• Poutine in Montreal!
https://channel9.msdn.com/Shows/Visual-Studio-Toolbox/App-Development-with-Cortana
More Cortana Dev on Windows 10
Thank You!
Slides are in SlideShare. Demos are on GitHub.
Contact me and let me know what you build, I will be happy to help promote your apps.
Blog: AgeofMobility.com
Twitter: @ActiveNick
Email: nick.landry@microsoft.com
Apps: www.bigbaldapps.com
LinkedIn: linkedin.com/in/activenick
GitHub: github.com/ActiveNick
Slideshare: slideshare.net/ActiveNick
Cognitive Services: Building Smart Apps with Speech, NLP & Vision

Cognitive Services: Building Smart Apps with Speech, NLP & Vision