the bing platform that powers cortana
savasp
http://savas.me
why a personal assistant?
cortana introduction
key scenarios and the cloud
lessons learnt
crazy scenarios we can build
increasing focus on information (& knowledge), personalization, context, wearables, …
anticipatory computing
importance of...
PERSONAL
Cortana…
…is your truly personal assistant
…gets to know you
…is transparent
Examples:
 Learning
 Notebook
 Pe...
bing platform as the foundation for
personal assistant experiences
user centric
cloud-driven
service-oriented
asynchronous...
ingredients – how to build a personal assistant
privacy/security
design
knowledge platform
machine learning platform
feedb...
the notebook
cortana home
rank
html
request
language
generation
bing knowledge
?retrieve the user’s profile,
context, pending questions...
flight BA 49 is delayed
notification
information streams: sports, flights,
weather, traffic, news, packages,
user location...
is this your home?
commute to work around
8.30am on Mondays
inferences
user understanding/insight
“remind me to wish Paul
happy birthday”
speech recognition
natural language
understanding
conversation
management
represen...
“will I need a scarf
tomorrow?”
speech recognition
natural language
understanding
conversation
management
user
profile/con...
bing.com
rank
html
request
language
generation
bing knowledge
?retrieve the user’s profile,
context, pending questions
agg...
ideas
DISCLAIMER: what follows does NOT represent
future products or services by microsoft
while at Sydney 2011, you should talk to john s.
both of you were at paris 2010, rio 2009, and
istanbul 2002 conferences
y...
“what was the title of the song
that got me dancing during the
radiohead concert few days ago?”
emergence of personal assistant as ux
metaphore, consolidation of user experiences
anticipatory computing, task completion...
Coldplay at the Gorge
2009 – 07 – 10, 9.12pm
Weather: Warm and mostly sunny
Attending: 20,000
Song playing at the time of ...
savas parastatidis
the bing platform that powers Cortana
savasp
http://savas.me
The Bing Platform that Powers Cortana
The Bing Platform that Powers Cortana
The Bing Platform that Powers Cortana
Upcoming SlideShare
Loading in...5
×

The Bing Platform that Powers Cortana

1,317
-1

Published on

High-level view of the Bing platform that powers the Cortana experiences.

Published in: Services
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,317
On Slideshare
0
From Embeds
0
Number of Embeds
9
Actions
Shares
0
Downloads
0
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide
  • Kinect avatars that are connected to a knowledge base
    Kids interact with the avatars and can ask questions
    “a virtual tutor, companion”
    Social experience… a virtual class
  • The Bing Platform that Powers Cortana

    1. 1. the bing platform that powers cortana savasp http://savas.me
    2. 2. why a personal assistant? cortana introduction key scenarios and the cloud lessons learnt crazy scenarios we can build
    3. 3. increasing focus on information (& knowledge), personalization, context, wearables, … anticipatory computing importance of near-real time processing/reactive computing transition from web to apps to personal assistants
    4. 4. PERSONAL Cortana… …is your truly personal assistant …gets to know you …is transparent Examples:  Learning  Notebook  Personal suggestions  Transparency & control LOOKS OUT FOR YOU Cortana… …looks out for you …filters out the noise …reminds you of what’s important Examples:  Useful and relevant alerts  Planners  Event scheduling  Quiet hours and inner circle  Reminders DELIGHTFUL & EASY TO USE Cortana… …“just works” …lets you interact on your terms …has a fun & engaging personality Examples:  Voice & natural language  Text input  Personality (visual, spoken voice, and behavior)   
    5. 5. bing platform as the foundation for personal assistant experiences user centric cloud-driven service-oriented asynchronous, reactive, functional, stateless data-driven configuration-driven secure and available
    6. 6. ingredients – how to build a personal assistant privacy/security design knowledge platform machine learning platform feedback loop infrastructure notifications infrastructure stream/complex event processing metrics/data-driven engineering speech recognition natural language understanding user understanding conversational & intent understanding personality global datacenter footprint legal business development …
    7. 7. the notebook
    8. 8. cortana home rank html request language generation bing knowledge ?retrieve the user’s profile, context, pending questions aggregate, filter inferences retrieve knowledge and rank based on the user’s current context
    9. 9. flight BA 49 is delayed notification information streams: sports, flights, weather, traffic, news, packages, user location, … notification information streams: sports, flights, weather, traffic, news, packages, user location, … var subscription = streamPlatform .GetObservable<FlightInfo>(Constants.FlightsStream) .Where(f => f.StatusCode == FlightStatus.Landed) .Select(f => string.Format("{0} {1}: {2}", f.Airline.Name, f.FlightNumber, f.StatusCode)) .Subscribe(...); stream processing platform
    10. 10. is this your home? commute to work around 8.30am on Mondays inferences user understanding/insight
    11. 11. “remind me to wish Paul happy birthday” speech recognition natural language understanding conversation management representation of intent speech stream Using Deep Neural Networks-trained models and other machine-learning techniques, we convert the voice stream to a representation the computers can understand We reason over the machine representation of the user’s intent. We use what we know about the user (e.g. “home” is transformed to an actual location) in order to fill in any gaps in our understanding. We maintain a dialog with the user (e.g. if the user had just said “remind me”, we would have initiated a follow up question). “when I get home, remind me to take out the garbage” language generation user profile/context web socket Using Deep Neural Networks-trained models and other machine-learning techniques, we convert the voice stream to a representation the computers can understand We reason over the machine representation of the user’s intent. We use what we know about the user (e.g. “home” is transformed to an actual location) in order to fill in any gaps in our understanding. We maintain a dialog with the user (e.g. if the user had just said “remind me”, we would have initiated a follow up question). “when I get home, remind me to take out the garbage” language generation user profile/context web socket mini reactor
    12. 12. “will I need a scarf tomorrow?” speech recognition natural language understanding conversation management user profile/context html speech stream Using Deep Neural Networks-trained models and other machine-learning techniques, we convert the voice stream to a representation the computers can understand We reason over the machine representation of the user’s intent. We consult Bing Knowledge and the user’s profile in order to construct a response in the conversation with the user. Since we are keeping the context of the conversation, the user can follow up with a question such as “How about next weekend?” or “What about Seattle?”. language generation bing knowledge rank
    13. 13. bing.com rank html request language generation bing knowledge ?retrieve the user’s profile, context, pending questions aggregate, filter inferences
    14. 14. ideas DISCLAIMER: what follows does NOT represent future products or services by microsoft
    15. 15. while at Sydney 2011, you should talk to john s. both of you were at paris 2010, rio 2009, and istanbul 2002 conferences you also seem to be reading the same journals on knowledge representation you both found the book “on intelligence” interesting finally, you both like sushi… i suggest “blue sushi”, just two blocks from the conference center
    16. 16. “what was the title of the song that got me dancing during the radiohead concert few days ago?”
    17. 17. emergence of personal assistant as ux metaphore, consolidation of user experiences anticipatory computing, task completion, conversational sensors, wearables reactive, near-realtime
    18. 18. Coldplay at the Gorge 2009 – 07 – 10, 9.12pm Weather: Warm and mostly sunny Attending: 20,000 Song playing at the time of photograph: Viva La Vida Did you know that it was Coldplay’s 1st appearance at the Gorge? Ad-hoc social streams: photos, tweets, emotions i
    19. 19. savas parastatidis the bing platform that powers Cortana savasp http://savas.me

    ×