Slides used during FraserHacks event about Cognitive Services and Hololens Demo in the Toronto University.

1. Sabrina Smai
Student Tech Evangelist @ Microsoft
@sabrina_smai
sabrina.smai@mail.utoronto.ca
Bruno Capuano
Microsoft MVP @ Avanade
@elbruno
http://www.elbruno.com

2. Microsoft
Cognitive
Services
Give your apps
a human side
From faces to feelings, allow your
apps to understand images and video
Hear and speak to your users by filtering noise,
identifying speakers, and understanding intent
Process text and learn how to
recognize what users want
Tap into rich knowledge amassed from
the web, academia, or your own data
Access billions of web pages, images, videos,
and news with the power of Bing APIs

3. Cortana Intelligence Suite
Intelligent Apps require Intelligent Solutions
Data
Sources
Apps
Sensors
and
devices
Data Action
People
Automated
Systems
Apps
Web
Mobile
Bots
Intelligence
Dashboards &
Visualizations
Information
Management
Big Data Stores Machine Learning
and Analytics
CortanaEvent Hub
HDInsight
(Hadoop and
Spark)
Stream Analytics
Intelligence
Bot
Framework
SQL Data
WarehouseData Catalog
Data Lake
Analytics
Data Factory
Machine
Learning
Data Lake Store
Blob Store
Cognitive
Services
Power BI

4. microsoft.com/cognitive

5. http://www.microsoft.com/cognitive

7. • Personalized, friendly, approachable, natural •
•
•
• Predictive, current, and familiar •
•
• Always available, always contextual •
•
•

9. Vision APIs
Analyze an Image
OCR (Optical Character Recognition)
Get Thumbnails

10. Analyze Image Service

11. OCR Service

12. OCR – Example
JSON:
{
"language": "en",
"orientation": "Up",
"regions": [
{
"boundingBox": "41,77,918,440",
"lines": [
{
"boundingBox": "41,77,723,89",
"words": [
{
"boundingBox": "41,102,225,64",
"text": "LIFE"
},
{
"boundingBox": "356,89,94,62",
"text": "IS"
},
{
"boundingBox": "539,77,225,64",
"text": "LIKE"
}
. . .
TEXT:
LIFE IS LIKE
RIDING A BICYCLE
TO KEEP YOUR BALANCE
YOU MUST KEEP MOVING

13. Face APIs
Detection
Verification
Grouping
Identification

14. Face API – Detection
Detection Result:
JSON:
[
{
"faceRectangle": {
"width": 109,
"height": 109,
"left": 62,
"top": 62
},
"attributes": {
"age": 31,
"gender": "male",
"headPose": {
"roll": "2.9",
"yaw": "-1.3",
"pitch": "0.0"
}
"faceLandmarks": {
"pupilLeft": {
"x": "93.6",
"y": "88.2"
},
"pupilRight": {
"x": "138.4",
"y": "91.7"
},
...

15. Verification Result:
JSON:
[
{
"isIdentical":false,
"confidence":0.01
}
]
Face API – Verification

16. Face API – Grouping

17. Start coding

20. Language Understanding
Intelligent Service
Determine Intent
Detect Entities
Improve Models

21. Language Understanding Intelligent Service

22. NYT: I am feeling down. I’ve been having a
tough time at work lately.
Hannah: If work were going well and you had
a great relationship, you wouldn’t get the
chance to drink with people.
NYT: I don’t want to drink with others. I only
want to talk to you.
Hannah: Who do you like to talk to? (emoji
signals “none of my business”)
NYT: You. I feel that you are the only person
that gets me.
Hannah: Ha-ha/So I’m getting you?
NYT: You are the only person wo chats with
me.
Hannah: It’s not what you talk about that’s
important, but who you talk with.

23. http://cortanaintelligence.com
http://gallery.cortanaintelligence.com
http://www.microsoft.com/cognitive

24. Sabrina Smai
Student Tech Evangelist @ Microsoft
@sabrina_smai
sabrina.smai@mail.utoronto.ca
Bruno Capuano
Microsoft MVP @ Avanade
@elbruno
http://www.elbruno.com