SlideShare a Scribd company logo
1 of 21
Download to read offline
Taking the Road Less Travelled:
In pursuit of a Multi-modal
experience for Bixby
Samsung R&D Bangalore, India
Dr. Vikram Vij
vikram.v@samsung.com
2013
Team Foundation
for Voice
Intelligence
2015
US Launch Voice
Assistance
2017
Bixby Global
Launch
SRI-B’s Foot Steps Towards
Voice Intelligence
Bixby Introduction
• Bixby is an intelligent, personalized voice interface for your phone.
• It lets you seamlessly switch between voice and touch modes.
• Just speak using natural language and Bixby will execute your command.
• Launch Date : 19th July 2017 (US), 22nd Aug (Global)
• Expanded in more than 200 countries
• More than 75 Domains supported (eg. Camera, Gallery, Messages, WhatsApp,
Youtube, Uber etc.)
• http://bixby.samsung.com/meet-bixby
• http://bixby.samsung.com/philosophy
• https://www.youtube.com/watch?v=Gcd4NpK2fTI
Bixby | Human Computer Interface Revolution
With English Support, Samsung's Bixby Impresses Vs. Siri And
Google Assistant
Bixby is perhaps in the most precarious spot, as it’s going to be competing
directly against Google Assistant on some devices. Bixby’s capabilities
sound quite impressive thanks to its integration with other Samsung apps
Galaxy S8's voice sidekick can do things Siri can't
Traditional Flow of Voice Assistance
NLU
Platform
mom
Text to Mom
Machine Learning Models
Command
Domain
Classifier
Intent
Classifier
Slot
Tagger
Bixby v1.0 in a Minimalistic View
ASR
NLU
voice packet
text input
command
Bixby ASR - Fundamentals
Language
Model(s)
voice packet
Feature
Extraction Decoder
Acoustic
Model
ASR
System
ASR
Hypothesis
Inverse
Text
Normalization
Bixby ASR – Multi Accent
United States China
India
United Kingdom
SpainSouth Korea
DEFAULT ACCENTED
On-Boarding
Utterances
SIM Card
Information
Keyboard
Language
Contact
Details
Accent Determination
Australia
Canada
Bixby: The Multi-Modal Point of View
① Home ② Settings ③ Connections ③ Data Usage
Touch Interface Voice Interface
+
“Show me the mobile data usage”
Bixby: The Multi-Modal Point of View (cont’d)
Touch UI
Screen Flow
Voice UI
“Find Hawaii photos in Gallery”
Context Context Context Context
Leap Required for NLU toward Multi-Modality
Traditional
NLU
Multi-Modal
NLU
Context
Awareness
Massive Number of Contexts Varying Set of Commands
…
…
…
…
…
Thousands of states
Note8 …
…
…
…
…
…
S8
TabS
Various device models,
apps, locales, …
Challenge of Massive Contextual Input Space
Challenge of RNN vs CNN
Challenge of Filter Size
Data Governence
Approach for Massive Contextual Input Space
Challenge of Variable Output Space
Approach for Variable Output Space
Challenge for Indian Market
• Hindi targeted as language of experimentation.
• Indian Languages e.g. Hindi is used in conjunction with English
e.g. camera खुला करो
• We have developed bi-lingual model for Hindi classifier
Bixby1.0 is a Success
0
1000000
2000000
3000000
4000000
5000000
6000000
7000000
8000000
Aug'17 Sep'17 Oct'17 Nov'17 Dec'17 Jan'18 Feb'18
Total Bixby Users and Active Users
Total Users DAU
US Launch 19th July 2017
Global Launch 22nd Aug 2017
Samsung voice intelligence   outline

More Related Content

Similar to Samsung voice intelligence outline

Building speech enabled products with Amazon Polly & Amazon Lex
Building speech enabled products with Amazon Polly & Amazon LexBuilding speech enabled products with Amazon Polly & Amazon Lex
Building speech enabled products with Amazon Polly & Amazon Lex
Amazon Web Services
 
Building Speech Enabled Products with Amazon Polly & Amazon Lex
Building Speech Enabled Products with Amazon Polly & Amazon LexBuilding Speech Enabled Products with Amazon Polly & Amazon Lex
Building Speech Enabled Products with Amazon Polly & Amazon Lex
Amazon Web Services
 
For students of engineering in India
For students of engineering in IndiaFor students of engineering in India
For students of engineering in India
Samir Shah
 

Similar to Samsung voice intelligence outline (20)

Solvion Trendwerkstatt - Microsoft Azure + Bots
Solvion Trendwerkstatt - Microsoft Azure + BotsSolvion Trendwerkstatt - Microsoft Azure + Bots
Solvion Trendwerkstatt - Microsoft Azure + Bots
 
Getting Started with Amazon Lex - AWS Summit Cape Town 2017
Getting Started with Amazon Lex  - AWS Summit Cape Town 2017 Getting Started with Amazon Lex  - AWS Summit Cape Town 2017
Getting Started with Amazon Lex - AWS Summit Cape Town 2017
 
Alexa user group bangalore meetup - let us build multimodal skill
Alexa user group   bangalore meetup - let us build multimodal skillAlexa user group   bangalore meetup - let us build multimodal skill
Alexa user group bangalore meetup - let us build multimodal skill
 
Building speech enabled products with Amazon Polly & Amazon Lex
Building speech enabled products with Amazon Polly & Amazon LexBuilding speech enabled products with Amazon Polly & Amazon Lex
Building speech enabled products with Amazon Polly & Amazon Lex
 
BDA306 NEW LAUNCH! An Introduction to Amazon Lex, your service for building v...
BDA306 NEW LAUNCH! An Introduction to Amazon Lex, your service for building v...BDA306 NEW LAUNCH! An Introduction to Amazon Lex, your service for building v...
BDA306 NEW LAUNCH! An Introduction to Amazon Lex, your service for building v...
 
Designing XR Experiences with Speech & Natural Language Understanding in Unity
Designing XR Experiences with Speech & Natural Language Understanding in UnityDesigning XR Experiences with Speech & Natural Language Understanding in Unity
Designing XR Experiences with Speech & Natural Language Understanding in Unity
 
Bots: The Unspoken Challenge of Conversations
Bots: The Unspoken Challenge of ConversationsBots: The Unspoken Challenge of Conversations
Bots: The Unspoken Challenge of Conversations
 
Designing XR Experiences with Speech & Natural Language Understanding in Unity
Designing XR Experiences with Speech & Natural Language Understandingin UnityDesigning XR Experiences with Speech & Natural Language Understandingin Unity
Designing XR Experiences with Speech & Natural Language Understanding in Unity
 
Smart Speaker PPT G2.pptx
Smart Speaker PPT G2.pptxSmart Speaker PPT G2.pptx
Smart Speaker PPT G2.pptx
 
Building Speech Enabled Products with Amazon Polly & Amazon Lex
Building Speech Enabled Products with Amazon Polly & Amazon LexBuilding Speech Enabled Products with Amazon Polly & Amazon Lex
Building Speech Enabled Products with Amazon Polly & Amazon Lex
 
IBM Cloud Treehacks Tech Talk
IBM Cloud Treehacks Tech TalkIBM Cloud Treehacks Tech Talk
IBM Cloud Treehacks Tech Talk
 
Serve Your Customers with AI from the Cloud: AWS Developer Workshop - Web Sum...
Serve Your Customers with AI from the Cloud: AWS Developer Workshop - Web Sum...Serve Your Customers with AI from the Cloud: AWS Developer Workshop - Web Sum...
Serve Your Customers with AI from the Cloud: AWS Developer Workshop - Web Sum...
 
20160813 102-59-kim youngwook
20160813 102-59-kim youngwook20160813 102-59-kim youngwook
20160813 102-59-kim youngwook
 
Мобильная разработка. Между Сциллой и Харибдой. Native, hybrid or cross platf...
Мобильная разработка. Между Сциллой и Харибдой. Native, hybrid or cross platf...Мобильная разработка. Между Сциллой и Харибдой. Native, hybrid or cross platf...
Мобильная разработка. Между Сциллой и Харибдой. Native, hybrid or cross platf...
 
Azure Weekend 2020 Build Malaysia Bus Uncle Chatbot
Azure Weekend 2020 Build Malaysia Bus Uncle ChatbotAzure Weekend 2020 Build Malaysia Bus Uncle Chatbot
Azure Weekend 2020 Build Malaysia Bus Uncle Chatbot
 
Artifical Intelligence in Robotics
Artifical Intelligence in RoboticsArtifical Intelligence in Robotics
Artifical Intelligence in Robotics
 
Cloud-Native Roadshow Google Cloud Platform - Los Angeles
Cloud-Native Roadshow Google Cloud Platform - Los AngelesCloud-Native Roadshow Google Cloud Platform - Los Angeles
Cloud-Native Roadshow Google Cloud Platform - Los Angeles
 
For students of engineering in India
For students of engineering in IndiaFor students of engineering in India
For students of engineering in India
 
Conversational AI: What's New?
Conversational AI: What's New?Conversational AI: What's New?
Conversational AI: What's New?
 
re:Invent Recap keynote - An introduction to the latest AWS services
re:Invent Recap keynote  - An introduction to the latest AWS servicesre:Invent Recap keynote  - An introduction to the latest AWS services
re:Invent Recap keynote - An introduction to the latest AWS services
 

Recently uploaded

Call Girls in Karachi || 03274100048 || 50+ Hot Sexy Girls Available 24/7
Call Girls in Karachi || 03274100048 || 50+ Hot Sexy Girls Available 24/7Call Girls in Karachi || 03274100048 || 50+ Hot Sexy Girls Available 24/7
Call Girls in Karachi || 03274100048 || 50+ Hot Sexy Girls Available 24/7
Sana Rajpoot
 
Call Girls in Sialkot || 🥵👙 || 03280288848
Call Girls in Sialkot || 🥵👙 || 03280288848Call Girls in Sialkot || 🥵👙 || 03280288848
Call Girls in Sialkot || 🥵👙 || 03280288848
Ifra Zohaib
 
Low Rate Russian Call Girls In Lajpat Nagar ➡️ 7836950116 Call Girls Service ...
Low Rate Russian Call Girls In Lajpat Nagar ➡️ 7836950116 Call Girls Service ...Low Rate Russian Call Girls In Lajpat Nagar ➡️ 7836950116 Call Girls Service ...
Low Rate Russian Call Girls In Lajpat Nagar ➡️ 7836950116 Call Girls Service ...
riyasharma00119
 
Girls For Night in Islamabad | 03274100048 🔞
Girls For Night in Islamabad | 03274100048 🔞Girls For Night in Islamabad | 03274100048 🔞
Girls For Night in Islamabad | 03274100048 🔞
Ifra Zohaib
 
Call Girls in Rawalpindi | 🍆💦 03280288848
Call Girls in Rawalpindi | 🍆💦 03280288848Call Girls in Rawalpindi | 🍆💦 03280288848
Call Girls in Rawalpindi | 🍆💦 03280288848
Ifra Zohaib
 
Call Girls In Lahore || 03274100048 ||Lahore Call Girl Available 24/7
Call Girls In Lahore || 03274100048 ||Lahore Call Girl Available 24/7Call Girls In Lahore || 03274100048 ||Lahore Call Girl Available 24/7
Call Girls In Lahore || 03274100048 ||Lahore Call Girl Available 24/7
Sana Rajpoot
 

Recently uploaded (20)

Nagpur ❤CALL GIRL 9874883814 ❤CALL GIRLS IN nagpur ESCORT SERVICE❤CALL GIRL I...
Nagpur ❤CALL GIRL 9874883814 ❤CALL GIRLS IN nagpur ESCORT SERVICE❤CALL GIRL I...Nagpur ❤CALL GIRL 9874883814 ❤CALL GIRLS IN nagpur ESCORT SERVICE❤CALL GIRL I...
Nagpur ❤CALL GIRL 9874883814 ❤CALL GIRLS IN nagpur ESCORT SERVICE❤CALL GIRL I...
 
Call Now ☎9870417354|| Call Girls in Noida Sector 12 Escort Service Noida N.C.R.
Call Now ☎9870417354|| Call Girls in Noida Sector 12 Escort Service Noida N.C.R.Call Now ☎9870417354|| Call Girls in Noida Sector 12 Escort Service Noida N.C.R.
Call Now ☎9870417354|| Call Girls in Noida Sector 12 Escort Service Noida N.C.R.
 
Call Girls in Karachi || 03274100048 || 50+ Hot Sexy Girls Available 24/7
Call Girls in Karachi || 03274100048 || 50+ Hot Sexy Girls Available 24/7Call Girls in Karachi || 03274100048 || 50+ Hot Sexy Girls Available 24/7
Call Girls in Karachi || 03274100048 || 50+ Hot Sexy Girls Available 24/7
 
Guwahati ❣️ Call Girl 97487*63073 Call Girls in Guwahati Escort service book now
Guwahati ❣️ Call Girl 97487*63073 Call Girls in Guwahati Escort service book nowGuwahati ❣️ Call Girl 97487*63073 Call Girls in Guwahati Escort service book now
Guwahati ❣️ Call Girl 97487*63073 Call Girls in Guwahati Escort service book now
 
Indore ❣️Call Girl 97487*63073 Call Girls in Indore Escort service book now
Indore  ❣️Call Girl 97487*63073 Call Girls in Indore Escort service book nowIndore  ❣️Call Girl 97487*63073 Call Girls in Indore Escort service book now
Indore ❣️Call Girl 97487*63073 Call Girls in Indore Escort service book now
 
Kolkata 💋 Call Girl 9748763073 Call Girls in Kolkata Escort service book now
Kolkata 💋 Call Girl 9748763073 Call Girls in Kolkata Escort service book nowKolkata 💋 Call Girl 9748763073 Call Girls in Kolkata Escort service book now
Kolkata 💋 Call Girl 9748763073 Call Girls in Kolkata Escort service book now
 
Call Girls in Sialkot || 🥵👙 || 03280288848
Call Girls in Sialkot || 🥵👙 || 03280288848Call Girls in Sialkot || 🥵👙 || 03280288848
Call Girls in Sialkot || 🥵👙 || 03280288848
 
Patna ❣️ Call Girl 7870993772 Call Girls in Patna Escort service book now
Patna ❣️ Call Girl 7870993772 Call Girls in Patna Escort service book nowPatna ❣️ Call Girl 7870993772 Call Girls in Patna Escort service book now
Patna ❣️ Call Girl 7870993772 Call Girls in Patna Escort service book now
 
Bhopal ❤CALL GIRL 9874883814 ❤CALL GIRLS IN Bhopal ESCORT SERVICE❤CALL GIRL I...
Bhopal ❤CALL GIRL 9874883814 ❤CALL GIRLS IN Bhopal ESCORT SERVICE❤CALL GIRL I...Bhopal ❤CALL GIRL 9874883814 ❤CALL GIRLS IN Bhopal ESCORT SERVICE❤CALL GIRL I...
Bhopal ❤CALL GIRL 9874883814 ❤CALL GIRLS IN Bhopal ESCORT SERVICE❤CALL GIRL I...
 
Varanasi Call Girl 78709*93772 Call Girls in Varanasi Escort service book now
Varanasi  Call Girl 78709*93772 Call Girls in Varanasi Escort service book nowVaranasi  Call Girl 78709*93772 Call Girls in Varanasi Escort service book now
Varanasi Call Girl 78709*93772 Call Girls in Varanasi Escort service book now
 
Pune ❤CALL GIRL 9874883814 ❤CALL GIRLS IN pune ESCORT SERVICE❤CALL GIRL IN We...
Pune ❤CALL GIRL 9874883814 ❤CALL GIRLS IN pune ESCORT SERVICE❤CALL GIRL IN We...Pune ❤CALL GIRL 9874883814 ❤CALL GIRLS IN pune ESCORT SERVICE❤CALL GIRL IN We...
Pune ❤CALL GIRL 9874883814 ❤CALL GIRLS IN pune ESCORT SERVICE❤CALL GIRL IN We...
 
NAGPUR CALL GIRL 7857803690 LOW PRICE ESCORT SERVICE
NAGPUR CALL GIRL 7857803690  LOW PRICE  ESCORT SERVICENAGPUR CALL GIRL 7857803690  LOW PRICE  ESCORT SERVICE
NAGPUR CALL GIRL 7857803690 LOW PRICE ESCORT SERVICE
 
9999266834 Call Girls In Noida Sector 37 (Delhi) Call Girl Service
9999266834 Call Girls In Noida Sector 37 (Delhi) Call Girl Service9999266834 Call Girls In Noida Sector 37 (Delhi) Call Girl Service
9999266834 Call Girls In Noida Sector 37 (Delhi) Call Girl Service
 
Low Rate Russian Call Girls In Lajpat Nagar ➡️ 7836950116 Call Girls Service ...
Low Rate Russian Call Girls In Lajpat Nagar ➡️ 7836950116 Call Girls Service ...Low Rate Russian Call Girls In Lajpat Nagar ➡️ 7836950116 Call Girls Service ...
Low Rate Russian Call Girls In Lajpat Nagar ➡️ 7836950116 Call Girls Service ...
 
Girls For Night in Islamabad | 03274100048 🔞
Girls For Night in Islamabad | 03274100048 🔞Girls For Night in Islamabad | 03274100048 🔞
Girls For Night in Islamabad | 03274100048 🔞
 
Call Girls in Rawalpindi | 🍆💦 03280288848
Call Girls in Rawalpindi | 🍆💦 03280288848Call Girls in Rawalpindi | 🍆💦 03280288848
Call Girls in Rawalpindi | 🍆💦 03280288848
 
Call Girls In Lahore || 03274100048 ||Lahore Call Girl Available 24/7
Call Girls In Lahore || 03274100048 ||Lahore Call Girl Available 24/7Call Girls In Lahore || 03274100048 ||Lahore Call Girl Available 24/7
Call Girls In Lahore || 03274100048 ||Lahore Call Girl Available 24/7
 
Mysore Call girl service 6289102337 Mysore escort service
Mysore Call girl service 6289102337 Mysore escort serviceMysore Call girl service 6289102337 Mysore escort service
Mysore Call girl service 6289102337 Mysore escort service
 
Indore Call girl service 6289102337 indore escort service
Indore Call girl service 6289102337 indore escort serviceIndore Call girl service 6289102337 indore escort service
Indore Call girl service 6289102337 indore escort service
 
Hyderabad ❣️ Call Girl 9748763073 Call Girls in Hyderabad Escort service boo...
Hyderabad ❣️  Call Girl 9748763073 Call Girls in Hyderabad Escort service boo...Hyderabad ❣️  Call Girl 9748763073 Call Girls in Hyderabad Escort service boo...
Hyderabad ❣️ Call Girl 9748763073 Call Girls in Hyderabad Escort service boo...
 

Samsung voice intelligence outline

  • 1. Taking the Road Less Travelled: In pursuit of a Multi-modal experience for Bixby Samsung R&D Bangalore, India Dr. Vikram Vij vikram.v@samsung.com
  • 2. 2013 Team Foundation for Voice Intelligence 2015 US Launch Voice Assistance 2017 Bixby Global Launch SRI-B’s Foot Steps Towards Voice Intelligence
  • 3. Bixby Introduction • Bixby is an intelligent, personalized voice interface for your phone. • It lets you seamlessly switch between voice and touch modes. • Just speak using natural language and Bixby will execute your command. • Launch Date : 19th July 2017 (US), 22nd Aug (Global) • Expanded in more than 200 countries • More than 75 Domains supported (eg. Camera, Gallery, Messages, WhatsApp, Youtube, Uber etc.) • http://bixby.samsung.com/meet-bixby • http://bixby.samsung.com/philosophy • https://www.youtube.com/watch?v=Gcd4NpK2fTI
  • 4. Bixby | Human Computer Interface Revolution With English Support, Samsung's Bixby Impresses Vs. Siri And Google Assistant Bixby is perhaps in the most precarious spot, as it’s going to be competing directly against Google Assistant on some devices. Bixby’s capabilities sound quite impressive thanks to its integration with other Samsung apps Galaxy S8's voice sidekick can do things Siri can't
  • 5. Traditional Flow of Voice Assistance NLU Platform mom Text to Mom Machine Learning Models Command Domain Classifier Intent Classifier Slot Tagger
  • 6. Bixby v1.0 in a Minimalistic View ASR NLU voice packet text input command
  • 7. Bixby ASR - Fundamentals Language Model(s) voice packet Feature Extraction Decoder Acoustic Model ASR System ASR Hypothesis Inverse Text Normalization
  • 8. Bixby ASR – Multi Accent United States China India United Kingdom SpainSouth Korea DEFAULT ACCENTED On-Boarding Utterances SIM Card Information Keyboard Language Contact Details Accent Determination Australia Canada
  • 9. Bixby: The Multi-Modal Point of View ① Home ② Settings ③ Connections ③ Data Usage Touch Interface Voice Interface + “Show me the mobile data usage”
  • 10. Bixby: The Multi-Modal Point of View (cont’d) Touch UI Screen Flow Voice UI “Find Hawaii photos in Gallery” Context Context Context Context
  • 11. Leap Required for NLU toward Multi-Modality Traditional NLU Multi-Modal NLU Context Awareness Massive Number of Contexts Varying Set of Commands … … … … … Thousands of states Note8 … … … … … … S8 TabS Various device models, apps, locales, …
  • 12. Challenge of Massive Contextual Input Space
  • 16. Approach for Massive Contextual Input Space
  • 17. Challenge of Variable Output Space
  • 18. Approach for Variable Output Space
  • 19. Challenge for Indian Market • Hindi targeted as language of experimentation. • Indian Languages e.g. Hindi is used in conjunction with English e.g. camera खुला करो • We have developed bi-lingual model for Hindi classifier
  • 20. Bixby1.0 is a Success 0 1000000 2000000 3000000 4000000 5000000 6000000 7000000 8000000 Aug'17 Sep'17 Oct'17 Nov'17 Dec'17 Jan'18 Feb'18 Total Bixby Users and Active Users Total Users DAU US Launch 19th July 2017 Global Launch 22nd Aug 2017