The document discusses Microsoft's cognitive services which provide various AI capabilities like vision, speech, language, knowledge and search. It describes capabilities for facial recognition, emotion detection, image analysis, language translation and understanding, intelligent search of web, images, videos and news. Microsoft's goal is to make these AI services easily accessible to developers through REST APIs and SDKs to build intelligent apps and chatbots.

7. BoT

8. BoT

9. 智慧服務
儀錶板 & 資料視覺化
巨量資料儲存 機器學習跟分析
行動
People
Automated
Systems
Apps
Web
Mobile
Bots
Cortana
Bot
Framework
Cognitive
Services
Power BI
資訊管理
Event Hubs
Data Catalog
Data Factory
HDInsight
(Hadoop and
Spark)
Stream
Analytics
智慧服務
Data Lake
Analytics
Machine
Learning
SQL Data
Warehouse
Data Lake
Store
資料
來源
應用
程式
感知器
與裝置
資料
IoT Hub

10. Bot Framework

11. 1
2
3
4
5
6
7
8
9
10
每月熱門APP
資料來源 AppAnnie 2015
1bn
100m100m
300m
1bn
總註冊使用者
(行動裝置及電腦2016)
217m

15. 在智慧型的互動模式中, 人跟機器
可以透過自然人類語言做順暢的溝
通
…
…
• 能發掘出語言中提到的人事時地物(Contextual
awareness)
• 記住跟每個人的聊天歷程
• 推敲出語句背後要表達的用意(Intention)
• 能了解互動當時的人跟情境, 並能有敏感度的回
應
發展對話機器人的挑戰

16. 微軟正在建造最全面的工具及服務給對話機器人(Bot)應用
產品與服務 | 個性
機器學習APIs:
電腦視覺, 語音, 語言, 知識, 搜尋, 翻譯
接在熟悉的平台上利用即時語音, 影像, 訊息, 群組訊息去接觸客戶
包括開發對話機器人的SDK, 讓對話機器人更聰明的感知服務與機器學習

17. 建造一次, 發佈到多個平台並容易被搜尋使用
提供開源(Open Source)的SDK讓開發者
更容易建置對話機器人, 數分鐘內就可以
建立完成
連接對話機器人到所有主流APP社交平
台, 可以接觸數十億使用者
讓對話機器人容易被使用者搜尋及找到

18. www.botframework.com

20. https://github.com/Microsoft/BotBuilder

22. Cognitive Services

23. Microsoft
感知服務
讓您的 Apps 了解圖片與影片的內容
讓您的 Apps 過濾雜訊可以辨識說話者是誰? 並
了解用戶的意圖
文字語意分析，讓 Apps 了解用戶文字的意涵
彙整來自互聯網，學術研究與您自己
所累積的知識與資料
透過 Bing API 存取數十億網頁, 圖片, 影片與
新聞

24. microsoft.com/cognitive

27. Intelligent Services Made Easy
語音時間長短 < 15 秒 < 2 分鐘
轉換成果
n-best 可供選
擇
句子停頓處提供最佳選擇
********* Final N-BEST Results *********
[0] Confidence=Normal Text="450 six St San Francisco."
[1] Confidence=Normal Text="For 50 six St San Francisco."
[2] Confidence=Normal Text="456th St San Francisco."
[3] Confidence=Normal Text="450 six St in San Francisco."
[4] Confidence=Normal Text="456 St San Francisco."
語音辨識模型 Voice Recognition Modes
Short Form Long Form
450 6th St.
San Francisco

28. 支援 REST API 與特定平台函式庫
real-time streaming

29. LUIS
了解對方在講些什麼
使用預設Bing 或 Cortana 模型或是
使用自己的模型

30. 理解人類鍵入之文字
提供互動式模型訓練工具漸少標示關鍵實體成本
呼叫內容自動保存，可日後持續訓練與學習
支援五種語言 (英文,中文, 義大利文, 法文, 西班牙文)
語言理解模型

31. microsoft.com/cognitive

32. LUIS (語言理解智能服務) - 讓機器聽得懂!
• 創建企業專屬的語言理解模型:

33. 示範例句
• 請問這裡是否有賣XXX(暖暖包)?!
模型訓練

34. 訓練成果
• 來一句新的：這裡還有沒有雷神巧克力?!

35. *screenshots show actual search results on bing.com
https://bingapis.azure-api.net/v5/search?q=nasa
{
“_type”: “SearchResponse”,
“queryContent”: {…},
“webPages”: {…},
“news”: {…},
“images”: {…},
“videos”: {…},
“relatedSearches”: {…},
“rankingResponse”: {…}
}
{
“answerType”:”WebPages”,
“resultIndex”:0,…
},
{
“answerType”:”News”,
“resultIndex”:1,…
}
Ranking Response
Search Response
Web
Results
Deep
Links
(1st Algo)
News
Results
Image
Results
Video
Results
Related
Searches

36. https://bingapis.azure-api.net/v5/images/search?q=shuttle+launch
• 支援多種過濾條件(尺寸, 版權, 色彩, 過濾成人內容 …等)
• Image insights(相似圖片，相關搜尋 … 等)
source: nasa.gov
https://bingapis.azure-api.net/v5/videos/search?q=viral+videos
• 支援多種過濾條件 (影片解析度,長度 … 等)
• 影片預覽圖片
https://bingapis.azure-api.net/v5/news/search?q=cuba
• 新聞分類
• 豐富之metadata (美國區域)
source: youtube.com
*screenshots show actual search results in bing.com
source: cnn.com

37. AI 服務架構
Technology Storyboard
Social Media
Customer
Insights
Productivity Cloud
Smart
Agents
Skype
Dynamics
Marketing/AX
CRM Online
PowerAppsCognitive
Services
Personal Graph
Office Graph
Bing
PowerBI
Office 365
Sharepoint
Knowledge Capture
Knowledge Sharing
Customer Support Case Management
Conversational AI Coaching
Intelligent Cloud
App Insights
HDInsight
Azure Machine
Learning
Data Factory
Data Lake
Stream
Analytics
Event Hubs
HDStormAzure SQL DB
Notification
Hubs
Microsoft
Social
Engagement
More Personal
Computing
Customer
Windows 10
Cortana
In-Store
Online
(web, chat)
Devices
Digital
Signage/
Kiosk
Equipment
Insights
Usage
Insights
Real-time contextual
data Weather, Events
& environmental
conditions
Real-Time network,
infrastructure &
customer telemetry
Customer Service
Engineering Knowledge Capture
Knowledge Sharing
Technical Case Management

39. Cognitive
Services
Give your solutions
a human side

40. Cognitive
Services
Give your solutions
a human side

41. Vision

44. JSON:
{
"language": "en",
"orientation": "Up",
"regions": [
{
"boundingBox": "41,77,918,440",
"lines": [
{
"boundingBox": "41,77,723,89",
"words": [
{
"boundingBox": "41,102,225,64",
"text": "LIFE"
},
{
"boundingBox": "356,89,94,62",
"text": "IS"
},
{
"boundingBox": "539,77,225,64",
"text": "LIKE"
}
. . .

47. "faceRectangle": {"width": 193,
"height": 193,
"left": 326,
"top": 204}
“scores": { "anger": 5.182241e-8,
"contempt": 0.0000242813,
"disgust": 5.621025e-7,
"fear": 0.00115027453,
"happiness": 1.06114619e-8,
"neutral": 0.003540177,
"sadness": 9.30888746e-7,
"surprise": 0.9952837}

49. "faceRectangle": {"width": 193, "height": 193, "left": 326, "top": 204}
…
"attributes": { "age": 42, "gender": "male",
"headPose": { "roll": "8.2", "yaw": "-37.8", "pitch": "0.0" }}

54. Speech

63. Language

65. 

68. Language understanding models

73. Knowledge

78. Search

79. Bing web search
Intelligent search to your apps with the ability to
comb billions of webpages, images, videos, and
news with a single API call
Retrieve web documents indexed by Bing and narrow the
results down with filters such as by answer type and freshness

80. Bing image search
A variety of image search options, from
trending images to detailed insights
Scour the web for images and get results that include
thumbnail and full image URLs, publishing website,
image metadata, related images, and more

81. Bing video search
A variety of advanced video search features,
including trending videos, price, and other
useful metadata
Find videos from across the web and get responses that provide
useful metadata including creator, encoding format, video size
and quality, and source view count

82. Bing news search
Turn any app into a news desk with world news
grouped and filtered by topic, local news, and
metadata you can mine

83. Bing auto suggest
Query completion suggestions
capabilities, so users can type less
and get to what they want faster
[