Probabilistic Retrieval Models - Sean Golliher Lecture 8 MSU CSCI 494

Probabilistic Retrieval Models Lecture 8 Sean A. Golliher

 Need to quickly cover some old material to understand the new methods

 Complex concept that has been studied for some time  Many factors to consider  People often disagree when making relevance judgments Retrieval models make various assumptions about relevance to simplify problem  e.g., topical vs. user relevance  e.g., binary vs. multi-valued relevance

 Older models  Boolean retrieval  Vector Space model Probabilistic Models  BM25  Language models Combining evidence  Inference networks  Learning to Rank

 Two possible outcomes for query processing  TRUE and FALSE  “exact-match” retrieval  simplest form of ranking Query usually specified using Boolean operators  AND, OR, NOT

 Advantages  Results are predictable, relatively easy to explain  Many different features can be incorporated  Efficient processing since many documents can be eliminated from search Disadvantages  Effectiveness depends entirely on user  Simple queries usually don’t work well  Complex queries are difficult

 Documents and query represented by a vector of term weights Collection represented by a matrix of term weights

 3-d pictures useful, but can be misleading for high-dimensional space

The Euclideandistance between qand d2 is large eventhough thedistribution of termsin the query q andthe distribution ofterms in thedocument d2 arevery similar.

 Thought experiment: take a document d and append it to itself. Call this document d′. “Semantically” d and d′ have the same content The Euclidean distance between the two documents can be quite large The angle between the two documents is 0, corresponding to maximal similarity (cos(0) = 1). Key idea: Rank documents according to angle with query.

 In Euclidean space, define dot product of vectors a & b asab=||a|| ||b|| cos a where b ||a|| == length == angle between a & b

 By using Law of Cosines, can compute coordinate-dependent definition in 3- space: ab= axbx + ayby + azbz cos = ab/||a|| ||b|| cosine(0) = 1 cosine(90 deg) = 0

 Documents ranked by distance between points representing query and documents  Similarity measure more common than a distance or dissimilarity measure  e.g. Cosine correlation

 Consider two documents D1, D2 and a query Q ○ D1 = (0.5, 0.8, 0.3), D2 = (0.9, 0.4, 0.2), Q = (1.5, 1.0, 0)

Dot product Unit vectors     V   q d q d q di i 1 i cos(q, d )     q d q d V 2 q V d i2 i 1 i i 1qi is the tf-idf weight of term i in the querydi is the tf-idf weight of term i in the documentcos(q,d) is the cosine similarity of q and d … or,equivalently, the cosine of the angle between q and d.

 tf.idf weight (older retrieval model)  tf: term frequency of term over collection of documents  idf: inverse document freq. ex: ○ log(N/n)  N is the total number of document  n is total number of documents that contain a term  Measure of “importance” of term. The more documents a term appears in the lest discriminating the term is.  Use log to dampen the effects

 The collection frequency of t is the number of occurrences of t in the collection, counting multiple occurrences. Word Collection frequency Document frequency insurance 10440 3997 try 10422 8760 Document frequency df is number of documents that contain a term t. Which of these is more useful?

 The tf-idf weight of a term is the product of its tf weight and its idf weight. Best known weighting scheme in information retrieval ○ Note: the “-” in tf-idf is a hyphen, not a minus sign! ○ Alternative names: tf.idf, tf x idf Increases with the number of occurrences within a document Increases with the rarity of the term in the collection

 Rocchio algorithm (paper topic) Optimal query  Maximizes the difference between the average vector representing the relevant documents and the average vector representing the non-relevant documents Modifies query according to  α, β, and γ are parameters ○ Typical values 8, 16, 4

 Most dominant paradigm used today Probability theory is a strong foundation for representing uncertainty that is inherent in IR process.

 Robertson (1977) If a reference retrieval system’s response to each request is a ranking of the documents in the collection in order of decreasing probability of relevance to the user who submitted the request, where the probabilities are estimated as accurately as possible on the basis of whatever data have been made available to the system for this purpose, the overall effectiveness of the system to its user will be the best that is obtainable on the basis of those data.”

 Probability Ranking Principle (Robertson, 70ies; Maron, Kuhns, 1959) Information Retrieval as Probabilistic Inference (van Rijsbergen & co, since 70ies) Probabilistic Indexing (Fuhr & Co.,late 80ies- 90ies) Bayesian Nets in IR (Turtle, Croft, 90ies) Probabilistic Logic Programming in IR (Fuhr & co, 90ies)

 P(a | b) => Conditional probability. Probability of a given that b occurred. Basic definitions (a È b) => AorB (a Ç b) = AandB

 Let D be a document in the collection. Let R represent relevance of a document w.r.t. given (fixed) query and let NR represent non-relevance. How do we find P(R|D)? Probability that a retrieved document is relevant. Abstract concept. P(R) is the probability that a retrieved is relevant Not Clear how to calculate this.

 Can we calculate P(D|R)? Probability of a document occurring in a set given a relevant set has been returned. If we KNOW we have relevant set of documents (maybe from humans?) We could calculate how often specific words occur in a certain set.

Let D be a document in the collection.Let R represent relevance of a document w.r.t. given (fixed)query and let NR represent non-relevance.Need to find p(R|D) - probability that a retrieved document Dis relevant. p(D | R)p(R) p(R),p(NR) - prior probability p(R | D) = p(D) of retrieving a (non) relevant p(xD | NR)p(NR) document p(NR | D) = p(xD) P(D|R), p(D|NR) - probability that if a relevant (non-rel document is retrieved, it is D.

 Bayes Decision Rule  A document D is relevant if P(R|D) > P(NR|D) Estimating probabilities  use Bayes Rule  classify a document as relevant if ○ Left side is likelihood ratio

 Can we calculate P(D|R)? Probability that if a relevant document is returned it is D? If we KNOW we have relevant set of documents (maybe from humans?) We could calculate how often specific words occur in a certain set. Ex: We have info on how often specific words occur in relevant set. We could calculate how likely it is to see the words appear in a set. Ex: Prob: “president” in the relevant set is 0.02 and “lincoln” in the relevant set is “0.03”. If new doc. has pres. & lincoln then prob. Is 0.02*0.03= .0006.

 Suppose we have a vector representing the presence and absence of terms (1,0,0,1,1). Terms 1, 4, & 5 are present. What is the probability of this document occurring in the relevant set? pi is the probability that the term i occurs in a relevant set. (1- pi ) would be the probability a term would not be included the relevant set. This gives us: p1 x (1-p2) x (1-p3) x p4 x p5

 Assume independence Binary independence model  Dot product of over terms that have value one. Zero means dot product over terms that have value 0.  pi is probability that term i occurs (i.e., has value 1) in relevant document, si is probability of occurrence in non-relevant document

 Scoring function is (Last term was same for all documents. So it can be ignored.

 Jump to machine learning and web search. Lots of training data available from web search queries. Learning to rank models. http://www.bradblock.com/A_General_La nguage_Model_for_Information_Retrieval .pdf

Probabilistic Retrieval Models - Sean Golliher Lecture 8 MSU CSCI 494

by sgollihe

Accessibility

Categories

Upload Details

Usage Rights

Statistics