Deep Semantic Similarity Model (DSSM) is a deep learning technique that represents text strings in a continuous semantic space to model semantic similarity. DSSM maps queries and documents to feature vectors using a neural network and computes cosine similarity between the vectors to rank documents. DSSM is trained using word hashing, convolutional and max-pooling layers, and negative sampling to maximize the likelihood of clicked documents. Evaluation on search engine queries showed DSSM improved ranking quality over baselines, and it has also been applied to recommendations by modeling similarities across domains.

1. Deep Semantic Similarity Model
Po-Sen Huang, CIKM2013
Presenter: Shuai Zhang, CSE, UNSW

2. Content
Introduction
Framework of DSSM
Training Techniques
DSSM for Recommender System

3. Introduction
DSSM: stands for Deep Structured Semantic Model or
Deep Semantic Similarity Model
Is a deep neural network modelling technique for
representing text strings in a continuous semantic space
and modelling semantic similarity between two text
strings.
web search ranking question answering
knowledge inference image captioning
machine translation recommendation

4. Introduction
automatically generating image descriptions
Recommending target documents to be
of interest to a user based on a source
document that she is reading

5. Motivation
Fuzzy keyword matching
q: cold home remedy
Spelling correction
q: cold remeedies
Query alteration/expansion
q: flu treatment
Query/document semantic matching
q: how to deal with stuffy nose
best home
remedies for
cold and flu

6. Structure of DSSM
• Compute semantic similarity between two text strings X and Y
• Map X and Y to feature vectors in a latent semantic space via deep
neural net
• Compute the cosine similarity between the feature vectors

7. Structure of DSSM
1. Get the semantic representation of two vector
2. Normalize the two semantic vectors
3. Compute their similarity
4. Use semantic similarity to
rank documents
Semantic representation

8. Structure of DSSM

9. Structure of DSSM
Semantic Relevance Score between a query Q and a document D
x: input
y: output
l: hidden layer
f: activation function

10. Structure of DSSM
Supervised Model: We assume that a query is relevant to the documents
that are clicked on for that query.
The posterior probability of a document given a query:
Where γ is the smoothing factor in the SoftMax function.
D denotes the set of candidate documents to be ranked.

11. Structure of DSSM
Maximize the likelihood of the clicked documents P(D|Q)
Equivalently, we need to minimize the following loss function
The model is trained using gradient-based numerical optimization
algorithms

12. Training Techniques
Word Hashing: use sub-word unit (e.g., letter n-gram) as raw input to
handle very large vocabulary,
Letter-trigram Representation
cat → #cat# → #-c-a, c-a-t, a-t-#
Only around 50K letter-trigrams in English
Advantages
• Capture sub-word semantics
• Control the dimensionality of the input space
• Words with small typos have similar raw representations

13. Training Techniques
Convolutional and Max-pooling layer: identify key words or concepts
Extract local features
using convolutional layer
Generate global features
using max-pooling

14. Training Techniques
Negative Sampling
Where γ is the smoothing factor in the SoftMax function.
D denotes the set of candidate documents to be ranked.
Ideally, D should contain all possible documents.
In practice, we usually approximate D by including clicked document set D+
and some randomly selected documents
posterior
probability

15. Training Techniques

16. Evaluation Results
NDCG: Normalized Discounted Cumulative Gain
A measure of ranking quality
Measures the usefulness of a document based on its position in the result list
The evaluation data set contains
16510 English queries sampled
from one-year query log files of
a commercial search engine.

17. DSSM for Recommendation
DSPR: deep-semantic similarity-based personalized recommendation

18. DSSM for Recommendation
Multi-View Deep Neural Network for Cross Domain Recommendation:
• Search Engine logs
• News article
browsing history
• App download logs
• Movie/TV view logs

19. References
1. https://en.wikipedia.org/wiki/N-gram
2. https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/wsdm2015.v3.pdf
3. https://www.microsoft.com/en-us/research/project/dssm/
4. https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/cikm2013_DSSM_fullversion.pdf

20. Thanks!
Q & A