Latent semantic analysis (LSA) is a technique in natural
language processing, in particular in vectorial semantics,
of analyzing relationships between a set of documents and
the terms they contain by producing a set of concepts
related to the documents and terms.
Latent semantic analysis
Natural language processing (NLP) is a field of computer
science, artificial intelligence, and linguistics concerned with the
interactions between computers and human (natural) languages.
Natural language processing could be divided in 4 phases:
Machine learning based toolkit
for the processing of natural
LSA could be seen as a part of NLP
Apache OpenNLP usage examples:
Chunker - Parser
Before the lexical analysis is possible to
use a sentences analysis tool: sentence
detector (Apache OpenNLP).
Supervised machine learning concepts
(ex: wikipedia corpus)
Humans produce a finite set of
It represents the training set.
It can be seen as discrete
Machine learning algorithm
(ex:linear regretion, maximum
Machine produces a model.
It can be seen as a continuous function.
(ex: just a document)
Input data are taken
from an infinte set.
Machine, using model
and input, produces
the expected output.
LSA assumes that words that are close in
meaning will occur in similar pieces of text.
LSA is a method for discovering hidden
concepts in document data.
LSA key concepts
Set of documents, each
LSA algorithm takes docs and words and
evaluates vectors in a semantic vectorial
• A documents/words matrix
• Singular value decomposition (SVD)
Semantic vectorial space.
Word1 and word2 are close,
it means that their (latent)
meaning is related.
Doc1 Doc2 Doc3 Doc4
Word1 1 0 1 0
Word2 1 0 1 1
Word3 0 1 0 1
1: there are occurrences of
the i-word in the j-doc.
0: there are not occurrences
of the i-word in the j-doc.
The matrix dimension is very
big (thousands of
words, hundreds of
Matrix SVD decomposition
To reduce the matrix dimension
Semantic Vector or JLSI
• SVD decomposition.
• Build the vectorial
UIMA to manage the solution
Some snipptes and console commands
OpenNLP has a command line tool which is used to train the models.
Models and document
This snippet takes as inputs 4 files and it evaluates a new file sentence detected, tokenized and POSTtaggered.
Document that is
POSTaggered, and that
could be, for example,
indexed in a search
engine like Apache Solr.
Note that the lucene-core is
a hierarchical dependency.
.bat file to load the classpath
SemanticVectors has two main functions:
1. Building wordSpace models.
To build the wordSpace model Semantic Vector
needs indexes created by Apache Lucene.
2. Searching through the vectors in such models.
Es: Bible chapter Indexed by Lucene
1. Building wordSpace models using pitt.search.semanticvectors.LSA class from
the index created by Apache Lucene (from a bible chapter).
In this example the Bible
chapter contains 29
documents, and in total
there are 2460 terms.
Semantic Vector builds:
1. 29 vectors that represent the documents (docvector.bin)
2. 2460 vectors that represent the terms (termvector.bin)
This two files represent the wordSpace.
Note that could be also possible to use pitt.search.semanticvectors.BuildIndex class that use Random Projection
instead of LSA to reduce the dimensional representation.
2. Searching through docVector and termVector
2.1 Searching for Documents using Terms
Search for document vectors closest to the vector ”Abraham”:
2.2 Using a document file as a source of queries
Find terms most closely related to Chapter 1 of Chronicles:
2.3 Search a general word
Find terms most closely related to “Abraham”.
2.4 Comparing words
Compare “abraham” with “Isaac”.
Compare “abraham” with “massimo”.