Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Sentiment Analysis Using Solr

Solr is an open source, widely used, popular IR machine. It can be used for simple sentiment analysis and sentiment retrieval tool. Its multi-language analyzers together with UIMA (Unstructured Information Management Architecture) framework can be extended for sentiment extraction. Each sentence passes through a series of pluggable annotators. Entity and its associated polarity are detected for each sentence. Polarity of each sentence is stored into Solr index. Persistent model files can be created from training data and accessed at run time.

Sentiment Analysis Using Solr

  1. 1. By: Pradeep Pujari
  2. 2. Working mostly in Search domainSearch = IR + ML + NLPWho am I?Works for
  3. 3. Contributing to SolrSherlock- Open Source ProjectWho am I?http://solrsherlock.github.io/SolrSherlock/
  4. 4. What is Sentiment Analysis?A linguistic analysis technique that identifiesThe movie is great.The movie stars Mr. XThe movie is horrible.opinion early in a piece of text.
  5. 5. ChallengingToo easy Too hardDifficultymisclassificationWhat is Sentiment Analysis?
  6. 6. SentimentAnalysisNLPCognitive ScienceWhat is Sentiment Analysis?
  7. 7. Human can easily understandemotions.Can a machine be trained to do it?What is Sentiment Analysis?
  8. 8. Solr ?Http Request ServletAdminInterfaceUpdate ServletStandardRequestHandlerCustomRequestHandlerResponseWriterSolr CoreLuceneAnalysis UIMAconfig CachingUpdateHandler
  9. 9. Linguistics moduleStems, Lemmas and Synonymsmulti language capabilityCJKAnalyzer, UIMA AnalyzersUIMA integrationUpdateProcessorChainWhy Solr ?
  10. 10. Why Solr ?Extract domain specific entitiesand conceptsTime and CostSolr Set Up – 5 minsUIMA Annotators - 5 daysEnrich text, write to dedicated field
  11. 11. Tagging entities in review textUsecaseI wasnt really in the market for another tablet, but my girlfriend endedup getting one for me so she got me on this one. I would like to say thatthis tablet reminds me of the first Motorola Droid smartphone that cameout several years back. The phone jam packed a ton of bells & whistlesinto its hardware and software to give a lot of bang for your buck. Thisis what it feels like amazon has done with the Kindle Fire 8.9. They haveput a lot of advanced hardware and innovative software, so for theaverage user, specially someone who absorbs a lot of media, you get alot for the price. But just because you get a lot for the price, doesntmean it is without its flaws.
  12. 12. UsecaseConsumer feedback about productsWhich product features are more relevantPolarity
  13. 13. Digital SLR with Full 1080p HD VideoThere are many preprogrammed scene modesthat make this a very easy camera to use.The picture quality is beyond belief, andeven better for the price.Price:Usecase
  14. 14. Why UIMA ?UIMA Framework manages componentsand data flow – No codingDeploy pipeline of analysis enginesAEs wrap NLP algorithmsPersonPlaceorganizationLanguageDetectionAggregate analysis engineSentenceAnnotatorPOSAnnotatorNER
  15. 15. IndexLuceneSolr UpdateRequestProcessorSolrQParser DataSolr+UIMAUIMA AE
  16. 16. NLP+UIMAUse POS in query understandingboosting termsSynonym expansionExtract concepts/entitiesFaceting using entitiesIdentify places in queryand use spatial queries
  17. 17. Ideas: Sentiment Analysis AppIdentify Subjective Sentences from textRemove noisy sentences– Regex, conditional probabilityGraph min cut – LingPipeSubjectivity LexiconsDiscard Facts and Objective Sentences
  18. 18. SubjectivitydetectorSubjectiveObjectivePolarityClassifierIdeas: Sentiment Analysis App
  19. 19. Sentiments Intensity - SentiWordNetWordNet-Affect: WordNet +annotated conceptsIdeas: Sentiment Analysis AppHybrid model with adding dictionary
  20. 20. UpdateHandler withprocessor chainRemove DuplicatesprocessorLoggingprocessorCustom TransformprocessorIndexprocessorUpdate Processor ChainTextAnalyzersLuceneLucene IndexSentence DetectionprocessorSentiment ClassifierCompany NameAnnotatorSentiment ScoreprocessorProduct Reviews
  21. 21. http://lucene.apache.org/solr/http://uima.apache.org/http://alias-i.com/lingpipe/demos/tutorial/sentiment/read-me.htmlhttp://openie.cs.washington.edu/
  22. 22. Questions ?
  23. 23. Thank YouEmail: pradeepp@rocketmail.com

×