Your SlideShare is downloading. ×
0
Looking at the world through         NLP-colored glasses Dr. John Feland, CEO, Argus Insights                      Sentime...
If you can not measure it, you can                  not improve it.                                                       ...
Always Check your math         -my First Grade Teacher
Trust, but Verify         - Russian Proverb   (Доверяй, но проверяй)
Most Corporate Clients Treat Sentiment as a Black Box    Raw musings of                                                   ...
How we checked the math                                      Unnamed                                      sentiment       ...
Initial compared sentiment scores directly                          0.6                                     Actual Distrib...
Scaled so that SV distro matches User distro                              0.6                                         Actu...
Comparing actual to scored by product    Reviews                       Raw SV Score                                    Cor...
New sentiment models - where the math checks out• Argus Insights uses customer reported sentiment• Proven predictor of mar...
With great power comes     great responsibility                                                     -Voltaire     © 2011 A...
Questions?
Upcoming SlideShare
Loading in...5
×

World throughnlp coloredglasses-final

673

Published on

Argus Insights CEO's presentation from the 2011 Sentiment Analysis Symposium in San Francisco. John walks through why it is important to validate sentiment measurements of social media and gives an example from a leading vender where 1 + 1 do not e

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
673
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
5
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "World throughnlp coloredglasses-final"

  1. 1. Looking at the world through NLP-colored glasses Dr. John Feland, CEO, Argus Insights Sentiment Analysis Symposium Research Workshop 8 Nov 2011 Tweet it! @argusinsights
  2. 2. If you can not measure it, you can not improve it. -Lord Kelvin © 2011 Argus Insights, Inc. Confidential: Do Not Distribute Page 2
  3. 3. Always Check your math -my First Grade Teacher
  4. 4. Trust, but Verify - Russian Proverb (Доверяй, но проверяй)
  5. 5. Most Corporate Clients Treat Sentiment as a Black Box Raw musings of Numerical Market Magic consumers Sentiment Sentiment Analysis• Corporate Decision Makers blinded by precision• Using these numbers to drive performance• What is the impact of getting it wrong? © 2011 Argus Insights, Inc. Confidential: Do Not Distribute Page 5
  6. 6. How we checked the math Unnamed sentiment provider• Over 1900 user reviews taken from the Tablet Market• Sources include most of the major online retailers• Leveraged best-in-class sentiment provider – Scored entire verbatim using tuned sentiment model – Usage is outside of vendor’s recommended use• Surprised at the results © 2011 Argus Insights, Inc. Confidential: Do Not Distribute Page 6
  7. 7. Initial compared sentiment scores directly 0.6 Actual Distribution SV DistributionPercentage Distribution 0.5 0.4 0.3 0.2 0.1 0 -1 -0.75 -0.5 -0.25 0 0.25 0.5 0.75 1 Consumer Sentiment • All scales normalized (linearly) to -1 to +1 • SV Sentiment Score within Review Range 7% of the time • SV Sentiment within correct Positive, Neutral or Negative territory 51% of the time • SV results are biased towards the negative for Tablets © 2011 Argus Insights, Inc. Confidential: Do Not Distribute Page 7
  8. 8. Scaled so that SV distro matches User distro 0.6 Actual Distribution SV DistributionP{ercentage of Distribution 0.5 0.4 0.3 0.2 0.1 0 -1 -0.75 -0.5 -0.25 0 0.25 0.5 0.75 1 Consumer Sentiment• Trimmed and scaled SV results to match actual distribution• Signficant performance improvements – Correct Bin: 33% -> almost 5X improvement – Correct Sign: 74% -> almost 50% improvement © 2011 Argus Insights, Inc. Confidential: Do Not Distribute Page 8
  9. 9. Comparing actual to scored by product Reviews Raw SV Score Corrected SV Score 1 ASUS Eee Slate RIM PlayBook RIM PlayBook 2 Iconia Andr Tablet iPad 2 iPad 2 3 Motorola Xoom Iconia Andr Tablet Iconia Andr Tablet 4 iPad 2 Samsung Galaxy Tab 7 Motorola Xoom 5 RIM PlayBook Motorola Xoom Samsung Galaxy Tab 7 6 Samsung Galaxy Tab 7 ASUS Eee Slate ASUS Eee Slate 7 Iconia Win Tablet Iconia Win Tablet Iconia Win Tablet 8 HP TouchPad HP TouchPad HP TouchPad• Calculated sentiment could lead to the wrong competitive understanding• Selecting the wrong target can be disastrous for the enterprise• Responsibility to our clients demand transparency © 2011 Argus Insights, Inc. Confidential: Do Not Distribute Page 9
  10. 10. New sentiment models - where the math checks out• Argus Insights uses customer reported sentiment• Proven predictor of market performance• Full transparency to clients, enabling them to trust and verify © 2011 Argus Insights, Inc. Confidential: Do Not Distribute Page 10
  11. 11. With great power comes great responsibility -Voltaire © 2011 Argus Insights, Inc. Confidential: Do Not Distribute Page 11
  12. 12. Questions?
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×