Enabling Opinion-Driven Decision Making - Sentiment Analysis Innovation Summit


Published on

This talk was given at the Sentiment Analysis Innovation Summit on how to leverage large amounts of opinions to help users make decisions. Topics include methods to abstract out opinions, opinion-driven search engine and how FindiLike Hotel Search uses some of the state-of-the-art opinion-driven decision making tools.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Enabling Opinion-Driven Decision Making - Sentiment Analysis Innovation Summit

  1. 1. Enabling Opinion-Driven Decision Making Kavita Ganesan Founder, FindiLike LLC.
  2. 2. State of the Web… Unstructured opinions: -e-commerce sites -online directories -blog articles -review sites -social networking sites -travel sites, more…
  3. 3. State of the Web… Abundance of opinions…but rarely used to help users make better & faster decisions! Unstructured opinions: -e-commerce sites -online directories -blog articles -review sites -social networking sites -travel sites, more…
  4. 4. Classic example: Looking for a hotel… Shortlist hotels by date availability Shortlist hotels by price Find hotels with specific amenities • pool & spa Find hotels that fulfill specific opinion criteria •“Safe neighborhood” •“Clean” •“Comfortable beds” • Have to read 100s of reviews of different hotels • May visit different websites Hotels.com  TripAdvisor  Yelp  Hotels.com
  5. 5.  Users: Highly tedious process! • Users perform own data mining • Personally done it…not fun! • No draw for users to come back to your site  decide to use other sites next time  Companies: Users are straying away from your site • Taking too long to get information needed  use some other method of booking • Worse: Found deals elsewhere while reading reviews  especially true with modern day contextual advertising Problematic to users & organizations! Need to use available opinions in a more intelligent & efficient way to enable decision making!
  6. 6. In the research world …  Many methods to leverage opinions for decision making  Not really used by industry!...YET Opinion-driven decision making tools
  7. 7. Opinion-driven decision making tools Opinion-Driven Search Opinion Summarization Opinion Trend Visualization
  8. 8. • Help users find entities (e.g. products, people, businesses) based on opinion preferences • Limits set of choices in consideration from a large number options Opinion-driven decision making tools Opinion-Driven Search Opinion Summarization Opinion Trend Visualization
  9. 9. • Help users understand underlying opinions in large amounts of unstructured text • Whole range of formats to abstract out opinions Opinion-driven decision making tools Opinion-Driven Search Opinion Summarization Opinion Trend Visualization
  10. 10. • Helps users understand change in opinion over a period of time • E.g. Smartphone: • started out with positive sentiments • sentiments changed after recent firmware upgrade Opinion-driven decision making tools Opinion-Driven Search Opinion Summarization Opinion Trend Visualization
  11. 11.  Goal of task: • Rank/recommend entities • Based on how well a users “opinion requirements” match the unstructured opinions about a set of entities Opinion-driven search “lightweight, responsive screen” opinion requirements • User provided • System suggested • Various interface options Smartphones Opinions
  12. 12.  New task in literature [Ganesan & Zhai IRJ 2012] [Ganesan PhD Thesis, 2013] [Choi et. al WWW’12] [Choi et. al CIKM’12]  Referred to as “opinion-based entity ranking”  Practical Implementation 1. Put search over results of opinion summarization  Fairly involved  requires work on search & summarization  Can be more accurate than other methods 2. Treat as information retrieval problem [Ganesan & Zhai 2012] [Ganesan 2013] [Choi et. al 2012a] [Choi et. al 2012b]  Extend robust IR models for this task  Easily scales up & can be used with different domains Opinion-driven search
  13. 13.  Preference based search: find entities based on preferences • Buy LCD TV: user preferences - “rich picture” ,“no glare” • System takes in preferences and lists TV’s matching criteria  Search filters: users filter results by specific opinions • Laptop search: limit to “lightweight” laptops Restaurant search: limit to “authentic” restaurants • Can be used in conjunction with your current search engine  Opinions can be “suggested” or “user enters” • Can also think about this as another form of faceted navigation How can we use opinion-driven search?
  14. 14.  Search filters: users filter results by specific opinions How can we use opinion-driven search? Babies R Us Car Seat
  15. 15.  Contextual advertising: recommend entities with similar or different opinions Shopping for Laptop: • Recommend similar laptops to one being viewed • Viewing laptop with negative opinions: recommend laptops with positive opinions • Brings users closer to the preferred type of laptop How can we use opinion-driven search?
  16. 16.  Big area of study in the research world • Sentiment classification • Aspect-based summarization • Text summarization • Co-reference resolution  Many different formats [Kim et al 2011] • Can range from simple sentiment summary to unstructured textual summaries Opinion Summarization
  17. 17. Sentiment Summary -- -- + + + + source2 source1 source3 source4 source5 source6 source7 Final sentiment summary: (+ve/–ve) or (score 3/5) -whole documents -passages -sentences -phrases --
  18. 18.  Gives users high level overview of underlying sentiments • Similar to overall ratings see in e-commerce sites  Widely studied in literature [Pang et al. 2002] [Pang and Lee 2004; 2005] [Dave et al. 2003] [Turney 2002] [Turney and Littman 2003]… • Heuristic based approaches to supervised methods  Fairly easy to implement • With supervised machine learning models + training data  Issues to consider: • Lacks sufficient details E.g. “hotel may get overall positive rating with extremely bad service” • To make it useful: pair with other summarization methods Sentiment Summary
  19. 19. Aspect-Based Summaries Aspects Ratings Design Screen Sound Battery 4.0/5 2.0/5 3.0/5 4.5/5 smartphone xyz
  20. 20. Aspect-Based Summaries Feature/Aspect Identification Design, battery sound, screen Sentiment Prediction Battery life is great… +ve Long battery life… +ve Horrible sound quality  -ve Star rating? Score based approach? 4/5, 2/5 Percentage? 60% 40% Presentation Each step can be implemented in different ways with different levels of sophistication aggregate
  21. 21.  Gives more details than overall sentiment ratings  Widely studied in literature – hot topic for several years! [Lu et al 09 ] [Titov & McDonald 2008] [Hu & Liu 2004a] [Hu & Liu 2006] [Ku et al. 2006] [Popescu & Etzioni 2005] [Zhuang et al. 2006] • Different options for implementation • Tweaked as needed to fit domain needs Aspect-Based Summaries
  22. 22.  Issues to consider: • Finding aspects or features in each domain  Varies from domain to domain  E.g. Electronics: features for television vs. smartphones  How to find features in a general and scalable way? • Lacks detailed reasons  E.g. Smartphone screen received score of 2/5?  Why? Screen too small? Screen non-responsive? no way of knowing!  To give more info: complement this textual summaries Aspect-Based Summaries
  23. 23.  Gives more details than structured summaries  Easiest implementation method: • Traditional text summarization • Select few sentences from text to make up summary Textual Opinion Summaries The xyz smartphone is easy to use and has a very bright screen. I mostly love that screen is so responsive.
  24. 24.  Not recommended for opinions! [ Ganesan et al COLING’10 ] • Can introduce bias when selecting sentences  Can select sentences that don’t represent major opinions “Battery life is great (10), but this phone is too small for me (1)”  Miss out key opinions when not selecting enough sentences • Not suitable for smaller screens  verbose if you keep adding sentences Textual Opinion Summaries
  25. 25.  Over last few years, people have been looking into micro-summarization approaches [Yatani et al CHI’11] [Ganesan et al COLING’10] [Ganesan et al WWW’12] [Khabiri 2013, PhD Thesis] [Potthast & Becker ECIR’10]  Concise summaries that represent key opinions in large amounts of text Micropinion Summaries
  26. 26. Micropinion Summaries [Ganesan etalCOLING’10, Ganesan etalWWW’12] http://www.findilike.com/demo.jsp Summary generated on reviews of Acura 2007 from Edmunds.com -Concise and readable -Picks up on aspects naturally -Contains important details -Displays how many people said it / how many times appeared
  27. 27. Micropinion Summaries [Ganesan etalCOLING’10, Ganesan etalWWW’12] http://www.findilike.com/demo.jsp -display snippets -display whole passages -up to application
  28. 28.  Highly flexible • Adjust summaries to screen size  longer summaries for larger screens  shorter summaries for smaller screens • Limiting duplicates, increasing diversity • Don’t need to know aspects/features in advance  Generate summaries for arbitrary aspects  You can limit to certain aspects if needed (e.g. battery, sound) • Recent user study showed with micro-summaries: [Yatani et al CHI’11]  Users took significantly less time to decide on a restaurant compared to reading full reviews  Micro-summaries shown to be effective in decision making Micropinion Summaries
  29. 29.  Practical Implementation • Not too hard to implement with n-gram methods • More sophisticated approaches – keyphrase extraction, multi-sentence compression [Fillipova COLING’10] [Ganesan et al COLING’10] [Ganesan et al WWW’12] [Boudin & Morin 2013]  Issues to consider: • Simple methods: chances of generating junk • E.g. iPhone 5s: “the battery life is”, “the iPhone 5s”  meaningless to user • You want: “battery life is short”, “iPhone 5s is reliable” Micropinion Summaries
  30. 30.  Comparative summaries • compare contradicting opinions  Entity based summarization Other summarization formats
  31. 31.  Decision making tools shown: • Are powerful, but have their limitations • Just as in any new state-of-the art methods  Pair techniques wisely + take edge cases into consideration  powerful solution!  How FindiLike Hotel Search uses opinion-driven decision making tools Example Implementation
  32. 32.  Search system: Helps users find hotels by preferences • Unstructured opinion preferences “friendly service”, “clean”, “good views” • Common structured preferences price [$0-$100], distance [5 miles from campus]  Beyond search: Support for analysis of hotels • Micropinion summaries • Multi-word buzz phrases of reviews  Developed to showcase published ideas: related to enabling opinion-driven decision making [Ganesan et al COLING’10, Ganesan & Zhai IRJ, Ganesan & Zhai WWW’12, Ganesan 2013 PhD. Thesis] FindiLike Hotel Search
  33. 33. Let us say… • Visiting: Los Angeles • Main opinion criteria: hotels said to be clean & safe Input requirements as natural keywords
  34. 34. Hotels ranked by how well preferences are matched in opinions
  35. 35. • Preferences entered • How well preferences matched (stars) • Snippets to show why hotel was selected
  36. 36. More reviews on why the hotel matched ‘clean’
  37. 37. • Can add more preferences: ‘cheap’, ‘good room service’ • Remove preferences • Combine with structured preferences – price, distance Benefits of opinion-driven search: • Limits number of hotels in consideration • No need to read reviews to find hotels matching preferences
  38. 38. Micropinion summaries: • Concise opinion summaries • Highlights key opinions about hotel Benefit of micropinion summaries: • Helps understand underlying opinions within reviews • Further refine choices by knowing key opinions
  39. 39. Benefit of multi-word buzz phrase: • Helps users explore the opinion space. Learning through exploration • Extremely suitable when opinions are sparse Multi-word buzz phrases: • Highlights common phrases • Weighted by frequency and readability snippets related to “parking lot”
  40. 40. Benefit of multi-word buzz phrase: • Helps users explore the opinion space. Learning through exploration • Extremely suitable when opinions are sparse
  41. 41.  Improves user productivity • Eliminates need to read large number of opinions  Don’t need to perform own data mining  Helps user retention • Users can make better, faster and more informed decisions on your site  Likely to come back  Improves conversion rates • Users can make faster decisions  More likely to complete transaction on your site The need for opinion-driven applications
  42. 42. Are users asking for this? Yes! QUORA: Are there any service to summarize yelp reviews? I want to check the reviews of a roofing company but there are hundreds of them. Can I get some kind of summary (and maybe bar graph) without reading all of them?
  43. 43. Contact: Kavita Ganesan Email: kganes@findilike.com Personal web: kavita-ganesan.com Company web: findilike.com Thank you! Questions?