The document outlines Google's Smart Reply feature, which uses a data-driven sequence-to-sequence model to generate email responses. It discusses the challenges of understanding context and language, how the model is trained on email-reply pairs, and its deployment in Gmail for mobile replies. Despite its effectiveness, the model has limitations in matching tone and ensuring response diversity.

1. Confidential + Proprietary
Smart Reply: Learning a Model of
Conversation from Data
Anjuli Kannan
Software Engineer, Google Brain

2. Problem

3. Confidential + Proprietary
Can you do Tuesday or Wednesday?
Phil Sharp

4. Confidential + Proprietary
Tuesday Wednesday
Can you do Tuesday or Wednesday?
Phil Sharp

5. Smart Reply feature
● Provide text assistance for email
reply composition
● Targeted at mobile
● Responses can be sent on their
own or extended

6. Smart Reply feature predicts email responses
Smart Reply
Incoming
email
Response
email

7. Why is this task hard?
● extracting meaning from previous message
● generating language
● grammatical transformations between call and response
● matching style/tone

8. Why is this solution interesting?
● Model is learned fully from data

9. Model

10. Confidential + Proprietary
Neural network
Is a 4
Is a 5
...
...
Image: Wikipedia

11. Confidential + Proprietary
Neural network
Neuron
Is a 4
Is a 5

12. Confidential + Proprietary
Basic building block is the neuron
Greg Corrado

13. Confidential + Proprietary
Neural network
Is a 4
Is a 5
...
...

14. Confidential + Proprietary
Learn a function from one space to another
f(.)x ∈ Rn
y ∈ Rm

15. Confidential + Proprietary
Smartreply feature predicts email responses
Smartreply
Incoming
email
Response
email

16. Confidential + Proprietary
Recurrent neural networks handle sequences of input
Diagram by Felix Gers

17. Confidential + Proprietary
Recurrent neural networks handle sequences of input
Diagram by Felix Gers

18. Confidential + Proprietary
Recurrent neural networks handle sequences of input

19. Confidential + Proprietary
Reading a word into a feed-forward neural network
cat
output

20. Confidential + Proprietary
Reading a sequence of words into an RNN
That

21. Confidential + Proprietary
Reading a sequence of words into an RNN
That is

22. Confidential + Proprietary
Reading a sequence of words into an RNN
That is good

23. Confidential + Proprietary
Reading a sequence of words into an RNN
That is good !

24. Confidential + Proprietary
Reading a sequence of words into an RNN
That is good !
output

25. Sequence-to-sequence model
Sutskever et al, NIPS 2014

26. Sequence-to-sequence model
encoder decoder

27. Sequence-to-sequence model
Ingests incoming message Generates reply message

28. Inference

29. Reading a sequence of words into an RNN
How

30. Reading a sequence of words into an RNN
How are

31. Reading a sequence of words into an RNN
How are you

32. Reading a sequence of words into an RNN
How are you ?

33. Encoder ingests the incoming message
How are you ?
Internal state is a fixed
length encoding of the
message

34. Decoder is initialized with final state of encoder
How are you ? __

35. Decoder is initialized with final state of encoder
How are you ? __

36. Decoder predicts next word
How are you ? __

37. Decoder predicts next word
How are you ? ____ I

38. Smartreply model
How
Message

39. Smartreply model
How are
Message

40. Smartreply model
How are you
Message

41. Smartreply model
How are you ?
Message

42. Smartreply model
How are you ? __
I
Message
Response

43. Smartreply model
How are you ? __ I
I am
Message
Response

44. Smartreply model
How are you ? __ I am
I am great
Message
Response

45. Smartreply model
How are you ? __ I am great
I am great !
Message
Response
Vinyals & Le, ICML DL 2015

46. Inference
● Resulting model is fully generative
● Output distribution can be used to determine the most likely responses using a
beam search

47. Training

48. Training
● Training data is a corpus of email-reply pairs
● Both encoder and decoder are trained together (end-to-end)

49. Training
● Training data is a corpus of email-reply pairs
● Both encoder and decoder are trained together (end-to-end)

50. Confidential + Proprietary
Key points about model
● Everything is learned from data, even features
● Neural network smooths across language variation

51. Smart Reply in Production

52. Deployment & coverage
● Deployed in Inbox by Gmail
● Used to assist with more than 10% of all mobile replies

53. Examples

54. Quality
● How do we ensure that the response options are always high quality in content
and language?
○ Avoid incorrect grammar and mechanics, misspellings e.g., your the best
○ Avoid inappropriate, offensive responses. e.g., Leave me alone.
○ Deal with wide variability, informal language. e.g., got it thx
● Restricting model vocabulary is not sufficient!
Solution: Restrict to a fixed set of valid responses, derived automatically from data.

55. Most frequently used clusters

56. Confidential + Proprietary
What the model can do

57. Confidential + Proprietary
What the model can't do
● Match every user's tone and style

58. Confidential + Proprietary
What the model can't do
● Match every user's tone and style
● Ensure diverse options

59. Confidential + Proprietary
What the model can't do
● Match every user's tone and style
● Ensure diverse options
● Access and update any kind of state or knowledge base

60. Conclusions

61. Conclusions
● Sequence-to-sequence produces plausible email replies in many common
scenarios, when trained on an email corpus
● Smart Reply is deployed in Inbox by Gmail and generates more than 10% of
mobile replies

62. Confidential + Proprietary
Conclusions
● A conversation model learned entirely from data is very powerful
● A data-driven approach can be complementary to hand-crafted rules and
scenarios

63. Confidential + Proprietary
Collaborators
- Greg Corrado, Oriol Vinyals (Google Brain)
- Balint Miklos, Tobias Kaufman, Laszlo Lukacs, and Karol Kurach (GMail)
- Sujith Ravi (Google Research)

64. Confidential + Proprietary
Thank you!

65. Extra slides

66. Example

67. Unique cluster and suggestion usage

68. Ranking experiments