The document discusses various techniques for sentiment analysis, including feature selection techniques like pointwise mutual information (PMI), chi-square, TF-IDF, and association rule mining. It provides examples of features that can be extracted from text like names, terms, and relations. PMI is discussed in more detail, including how it is used to find collocations in text based on the information gained from the co-occurrence of words. Code in Python is also provided as an example of how to calculate PMI for bigrams and trigrams.

1. Sentiment Analysis
Techniques
Presented to: Prof. Dr. M.Abid Khan
Presented by: Asim Ullah Jan

2. Agenda
 Feature Selection Techniques
 PMI
 Chi-Square
 Association Rule Mining (Support, confidence, lift and Conviction)
 Latent Semantic Indexing
 TF-IDF
 Latent Dirichlet Allocation
 Classification Techniques
 Machine Learning
 Lexicon Based

3. Sentiment Analysis

4. Feature Extraction
 To recognize and classify significant vocabulary items in unrestricted natural
language texts.
 Let’s see an example…

5. Example of Vocabulary found
 Certificate of deposit
 CMOs
 Commercial bank
 Commercial paper
 Commercial Union
Assurance
 Commodity Futures
Trading Commission
 Consul Restaurant
 Convertible bond
 Credit facility
 Credit line
 Debt security
 Debtor country
 Detroit Edison
 Digital Equipment
 Dollars of debt
 End-March
 Enserch
 Equity warrant
 Eurodollar
 …

6. Extracted information categories
 Names of persons, organizations and places
 Multiword terms
 Abbreviations
 Relations
 Other useful stuff

7. Feature Selection
 Point-wise Mutual Information
 Latent Dirichlet Allocation
 Chi-Square
 TF-IDF
 Association Rule Mining

8. Point-wise Mutual Information
Agenda
 Definitions and Basics – Collocations + PMI
 Examples
 Problems
 PMI tasks
 PMI between 3 words

9. Collocations
 A collocation is an expression of two or more words that are some conventional
way of saying something
 Words together are more than the sum of their parts
 Usually cant be translated into another language word by word

10. Collocation Examples
 I’ll be in touch
 Light rain
 Hard-earned money
 Strong tea
 Fast food

11. Collocation
 Conditions for being a collocation
 Non-compositionality
 Non-substitutability
 Non-modifiability

12. Non-Compositionality
 The meaning of a collocation is not a compositions of the meaning of its parts
 Idioms
 “kick the bucket” mean “to die” not to literally kick buckets
 Non-idioms
 “Strong tea” does not mean the tea is physically strong

13. Non-Substitutability
 Cannot substitute synonyms for components of a collocation
 “White wine” cannot be replaces with “Yellow wine”
 “Fast food” and “Quick food”

14. Non-Modifiability
 Collocations cannot be freely modified with additional lexical material or
grammatical transformations
 “The cat has got your tongue” cannot be replaced with “the large, furry cat got
your tongue” or “the cat have your tongues”

15. Point-wise Mutual Information
 Information theory approach to find collocations
 Measure if how much one word tells us about the others. How much information we gain
 Can be negative or positive
 Given two events x’ and y’
 In this case x’ and y’ are occurrences of particular words

22. PMI code in Python
 import nltk
 from nltk.collocations import *
 bigram_measures = nltk.collocations.BigramAssocMeasures()
 trigram_measures = nltk.collocations.TrigramAssocMeasures()
 finder =
TrigramCollocationFinder.from_words(nltk.corpus.genesis.words('c:dd.txt'))
 print finder.nbest(trigram_measures.pmi, 20)

23. Output