World throughnlp coloredglasses-final
Upcoming SlideShare
Loading in...5
×
 

World throughnlp coloredglasses-final

on

  • 601 views

Argus Insights CEO's presentation from the 2011 Sentiment Analysis Symposium in San Francisco. John walks through why it is important to validate sentiment measurements of social media and gives an ...

Argus Insights CEO's presentation from the 2011 Sentiment Analysis Symposium in San Francisco. John walks through why it is important to validate sentiment measurements of social media and gives an example from a leading vender where 1 + 1 do not e

Statistics

Views

Total Views
601
Views on SlideShare
601
Embed Views
0

Actions

Likes
1
Downloads
4
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

World throughnlp coloredglasses-final World throughnlp coloredglasses-final Presentation Transcript

  • Looking at the world through NLP-colored glasses Dr. John Feland, CEO, Argus Insights Sentiment Analysis Symposium Research Workshop 8 Nov 2011 Tweet it! @argusinsights
  • If you can not measure it, you can not improve it. -Lord Kelvin © 2011 Argus Insights, Inc. Confidential: Do Not Distribute Page 2
  • Always Check your math -my First Grade Teacher
  • Trust, but Verify - Russian Proverb (Доверяй, но проверяй)
  • Most Corporate Clients Treat Sentiment as a Black Box Raw musings of Numerical Market Magic consumers Sentiment Sentiment Analysis• Corporate Decision Makers blinded by precision• Using these numbers to drive performance• What is the impact of getting it wrong? © 2011 Argus Insights, Inc. Confidential: Do Not Distribute Page 5
  • How we checked the math Unnamed sentiment provider• Over 1900 user reviews taken from the Tablet Market• Sources include most of the major online retailers• Leveraged best-in-class sentiment provider – Scored entire verbatim using tuned sentiment model – Usage is outside of vendor’s recommended use• Surprised at the results © 2011 Argus Insights, Inc. Confidential: Do Not Distribute Page 6
  • Initial compared sentiment scores directly 0.6 Actual Distribution SV DistributionPercentage Distribution 0.5 0.4 0.3 0.2 0.1 0 -1 -0.75 -0.5 -0.25 0 0.25 0.5 0.75 1 Consumer Sentiment • All scales normalized (linearly) to -1 to +1 • SV Sentiment Score within Review Range 7% of the time • SV Sentiment within correct Positive, Neutral or Negative territory 51% of the time • SV results are biased towards the negative for Tablets © 2011 Argus Insights, Inc. Confidential: Do Not Distribute Page 7
  • Scaled so that SV distro matches User distro 0.6 Actual Distribution SV DistributionP{ercentage of Distribution 0.5 0.4 0.3 0.2 0.1 0 -1 -0.75 -0.5 -0.25 0 0.25 0.5 0.75 1 Consumer Sentiment• Trimmed and scaled SV results to match actual distribution• Signficant performance improvements – Correct Bin: 33% -> almost 5X improvement – Correct Sign: 74% -> almost 50% improvement © 2011 Argus Insights, Inc. Confidential: Do Not Distribute Page 8
  • Comparing actual to scored by product Reviews Raw SV Score Corrected SV Score 1 ASUS Eee Slate RIM PlayBook RIM PlayBook 2 Iconia Andr Tablet iPad 2 iPad 2 3 Motorola Xoom Iconia Andr Tablet Iconia Andr Tablet 4 iPad 2 Samsung Galaxy Tab 7 Motorola Xoom 5 RIM PlayBook Motorola Xoom Samsung Galaxy Tab 7 6 Samsung Galaxy Tab 7 ASUS Eee Slate ASUS Eee Slate 7 Iconia Win Tablet Iconia Win Tablet Iconia Win Tablet 8 HP TouchPad HP TouchPad HP TouchPad• Calculated sentiment could lead to the wrong competitive understanding• Selecting the wrong target can be disastrous for the enterprise• Responsibility to our clients demand transparency © 2011 Argus Insights, Inc. Confidential: Do Not Distribute Page 9
  • New sentiment models - where the math checks out• Argus Insights uses customer reported sentiment• Proven predictor of market performance• Full transparency to clients, enabling them to trust and verify © 2011 Argus Insights, Inc. Confidential: Do Not Distribute Page 10
  • With great power comes great responsibility -Voltaire © 2011 Argus Insights, Inc. Confidential: Do Not Distribute Page 11
  • Questions?