SlideShare a Scribd company logo
1 of 12
Looking at the world through
         NLP-colored glasses
 Dr. John Feland, CEO, Argus Insights

                      Sentiment Analysis Symposium
                     Research Workshop 8 Nov 2011




                         Tweet it! @argusinsights
If you can not measure it, you can
                  not improve it.

                                                                  -Lord Kelvin




          © 2011 Argus Insights, Inc. Confidential: Do Not Distribute            Page 2
Always Check your math

         -my First Grade Teacher
Trust, but Verify
         - Russian Proverb
   (Доверяй, но проверяй)
Most Corporate Clients Treat Sentiment as a Black Box



    Raw musings of                                                        Numerical Market
                                     Magic
      consumers                                                             Sentiment
                                   Sentiment
                                    Analysis



• Corporate Decision Makers blinded by precision
• Using these numbers to drive performance


• What is the impact of getting it wrong?

                     © 2011 Argus Insights, Inc. Confidential: Do Not Distribute             Page 5
How we checked the math


                                      Unnamed
                                      sentiment
                                       provider




• Over 1900 user reviews taken from the Tablet Market
• Sources include most of the major online retailers
• Leveraged best-in-class sentiment provider
   – Scored entire verbatim using tuned sentiment model
   – Usage is outside of vendor’s recommended use
• Surprised at the results
                    © 2011 Argus Insights, Inc. Confidential: Do Not Distribute   Page 6
Initial compared sentiment scores directly
                          0.6
                                     Actual Distribution              SV Distribution
Percentage Distribution



                          0.5
                          0.4
                          0.3
                          0.2
                          0.1
                           0
                                -1   -0.75      -0.5         -0.25      0       0.25                         0.5   0.75   1
                                                                Consumer Sentiment

    • All scales normalized (linearly) to -1 to +1
    • SV Sentiment Score within Review Range 7% of the time
    • SV Sentiment within correct Positive, Neutral or Negative
      territory 51% of the time
    • SV results are biased towards the negative for Tablets

                                                   © 2011 Argus Insights, Inc. Confidential: Do Not Distribute                Page 7
Scaled so that SV distro matches User distro
                              0.6
                                         Actual Distribution               SV Distribution
P{ercentage of Distribution



                              0.5

                              0.4

                              0.3

                              0.2

                              0.1

                               0
                                    -1    -0.75       -0.5         -0.25      0       0.25                         0.5   0.75   1
                                                                      Consumer Sentiment

• Trimmed and scaled SV results to match actual
  distribution
• Signficant performance improvements
                              – Correct Bin: 33% -> almost 5X improvement
                              – Correct Sign: 74% -> almost 50% improvement

                                                     © 2011 Argus Insights, Inc. Confidential: Do Not Distribute                    Page 8
Comparing actual to scored by product
    Reviews                       Raw SV Score                                    Corrected SV Score
  1 ASUS Eee Slate                RIM PlayBook                                    RIM PlayBook
  2 Iconia Andr Tablet            iPad 2                                          iPad 2
  3 Motorola Xoom                 Iconia Andr Tablet                              Iconia Andr Tablet
  4 iPad 2                        Samsung Galaxy Tab 7                            Motorola Xoom
  5 RIM PlayBook                  Motorola Xoom                                   Samsung Galaxy Tab 7
  6 Samsung Galaxy Tab 7          ASUS Eee Slate                                  ASUS Eee Slate
  7 Iconia Win Tablet             Iconia Win Tablet                               Iconia Win Tablet
  8 HP TouchPad                   HP TouchPad                                     HP TouchPad



• Calculated sentiment could lead to the wrong
  competitive understanding
• Selecting the wrong target can be disastrous for the
  enterprise
• Responsibility to our clients demand transparency

                           © 2011 Argus Insights, Inc. Confidential: Do Not Distribute                   Page 9
New sentiment models - where the math checks out




• Argus Insights uses customer reported sentiment
• Proven predictor of market performance
• Full transparency to clients, enabling them to trust and verify
                      © 2011 Argus Insights, Inc. Confidential: Do Not Distribute   Page 10
With great power comes
     great responsibility

                                                     -Voltaire




     © 2011 Argus Insights, Inc. Confidential: Do Not Distribute   Page 11
Questions?

More Related Content

Viewers also liked

John Feland Thesis Defense Slides: 19 Nov 2004
John Feland Thesis Defense Slides: 19 Nov 2004John Feland Thesis Defense Slides: 19 Nov 2004
John Feland Thesis Defense Slides: 19 Nov 2004John Feland
 
Do Consumers Want High Performance Displays?
Do Consumers Want High Performance Displays?Do Consumers Want High Performance Displays?
Do Consumers Want High Performance Displays?John Feland
 
Touching Isn't It? Consumer Response to Touch and Gesture Technology for IHS...
Touching Isn't It?  Consumer Response to Touch and Gesture Technology for IHS...Touching Isn't It?  Consumer Response to Touch and Gesture Technology for IHS...
Touching Isn't It? Consumer Response to Touch and Gesture Technology for IHS...John Feland
 
Who Watches the Watchers? Analysis of Embedded Vision Adoption in Consumer El...
Who Watches the Watchers? Analysis of Embedded Vision Adoption in Consumer El...Who Watches the Watchers? Analysis of Embedded Vision Adoption in Consumer El...
Who Watches the Watchers? Analysis of Embedded Vision Adoption in Consumer El...John Feland
 
Argus snapshots for d3
Argus snapshots for d3Argus snapshots for d3
Argus snapshots for d3John Feland
 
What is driving engagement ahead of Mobile World Congress 2015?
What is driving engagement ahead of Mobile World Congress 2015?What is driving engagement ahead of Mobile World Congress 2015?
What is driving engagement ahead of Mobile World Congress 2015?John Feland
 
Solving the Sock Drawer Problem: How Are We Failing Wearables Consumers?
Solving the Sock Drawer Problem: How Are We Failing Wearables Consumers?Solving the Sock Drawer Problem: How Are We Failing Wearables Consumers?
Solving the Sock Drawer Problem: How Are We Failing Wearables Consumers?John Feland
 
Big Lies and Small Truths About Big Data
Big Lies and Small Truths About Big DataBig Lies and Small Truths About Big Data
Big Lies and Small Truths About Big DataJohn Feland
 
Big Lies and Small Truths, Tales From An Accidental Data Scientist
Big Lies and Small Truths, Tales From An Accidental Data ScientistBig Lies and Small Truths, Tales From An Accidental Data Scientist
Big Lies and Small Truths, Tales From An Accidental Data ScientistJohn Feland
 
IDEO Know How Talk 2 Dec 2004 regarding my PhD work
IDEO Know How Talk 2 Dec 2004 regarding my PhD workIDEO Know How Talk 2 Dec 2004 regarding my PhD work
IDEO Know How Talk 2 Dec 2004 regarding my PhD workJohn Feland
 
Lessons from Home Automation Early Adopters, Why Dropcam is better than Nest ...
Lessons from Home Automation Early Adopters, Why Dropcam is better than Nest ...Lessons from Home Automation Early Adopters, Why Dropcam is better than Nest ...
Lessons from Home Automation Early Adopters, Why Dropcam is better than Nest ...John Feland
 
Presentation at LTE Americas Conference on What Potential User Experiences wi...
Presentation at LTE Americas Conference on What Potential User Experiences wi...Presentation at LTE Americas Conference on What Potential User Experiences wi...
Presentation at LTE Americas Conference on What Potential User Experiences wi...John Feland
 
MBA: 9 шагов управление маркетингом
MBA: 9 шагов управление маркетингомMBA: 9 шагов управление маркетингом
MBA: 9 шагов управление маркетингомLLC Fiesta media
 
Section 1031 bb presentation 7 11 2013
Section 1031 bb presentation 7 11 2013Section 1031 bb presentation 7 11 2013
Section 1031 bb presentation 7 11 2013Rockland Trust
 
Chart Of Accounts
Chart Of AccountsChart Of Accounts
Chart Of AccountsSajid Ali
 

Viewers also liked (18)

John Feland Thesis Defense Slides: 19 Nov 2004
John Feland Thesis Defense Slides: 19 Nov 2004John Feland Thesis Defense Slides: 19 Nov 2004
John Feland Thesis Defense Slides: 19 Nov 2004
 
Do Consumers Want High Performance Displays?
Do Consumers Want High Performance Displays?Do Consumers Want High Performance Displays?
Do Consumers Want High Performance Displays?
 
Touching Isn't It? Consumer Response to Touch and Gesture Technology for IHS...
Touching Isn't It?  Consumer Response to Touch and Gesture Technology for IHS...Touching Isn't It?  Consumer Response to Touch and Gesture Technology for IHS...
Touching Isn't It? Consumer Response to Touch and Gesture Technology for IHS...
 
Who Watches the Watchers? Analysis of Embedded Vision Adoption in Consumer El...
Who Watches the Watchers? Analysis of Embedded Vision Adoption in Consumer El...Who Watches the Watchers? Analysis of Embedded Vision Adoption in Consumer El...
Who Watches the Watchers? Analysis of Embedded Vision Adoption in Consumer El...
 
Argus snapshots for d3
Argus snapshots for d3Argus snapshots for d3
Argus snapshots for d3
 
What is driving engagement ahead of Mobile World Congress 2015?
What is driving engagement ahead of Mobile World Congress 2015?What is driving engagement ahead of Mobile World Congress 2015?
What is driving engagement ahead of Mobile World Congress 2015?
 
Solving the Sock Drawer Problem: How Are We Failing Wearables Consumers?
Solving the Sock Drawer Problem: How Are We Failing Wearables Consumers?Solving the Sock Drawer Problem: How Are We Failing Wearables Consumers?
Solving the Sock Drawer Problem: How Are We Failing Wearables Consumers?
 
Big Lies and Small Truths About Big Data
Big Lies and Small Truths About Big DataBig Lies and Small Truths About Big Data
Big Lies and Small Truths About Big Data
 
Big Lies and Small Truths, Tales From An Accidental Data Scientist
Big Lies and Small Truths, Tales From An Accidental Data ScientistBig Lies and Small Truths, Tales From An Accidental Data Scientist
Big Lies and Small Truths, Tales From An Accidental Data Scientist
 
IDEO Know How Talk 2 Dec 2004 regarding my PhD work
IDEO Know How Talk 2 Dec 2004 regarding my PhD workIDEO Know How Talk 2 Dec 2004 regarding my PhD work
IDEO Know How Talk 2 Dec 2004 regarding my PhD work
 
Lessons from Home Automation Early Adopters, Why Dropcam is better than Nest ...
Lessons from Home Automation Early Adopters, Why Dropcam is better than Nest ...Lessons from Home Automation Early Adopters, Why Dropcam is better than Nest ...
Lessons from Home Automation Early Adopters, Why Dropcam is better than Nest ...
 
Presentation at LTE Americas Conference on What Potential User Experiences wi...
Presentation at LTE Americas Conference on What Potential User Experiences wi...Presentation at LTE Americas Conference on What Potential User Experiences wi...
Presentation at LTE Americas Conference on What Potential User Experiences wi...
 
MBA: 9 шагов управление маркетингом
MBA: 9 шагов управление маркетингомMBA: 9 шагов управление маркетингом
MBA: 9 шагов управление маркетингом
 
ADT Stack
ADT StackADT Stack
ADT Stack
 
Section 1031 bb presentation 7 11 2013
Section 1031 bb presentation 7 11 2013Section 1031 bb presentation 7 11 2013
Section 1031 bb presentation 7 11 2013
 
Khan Saeed Doc Files In PDF
Khan Saeed Doc Files In PDFKhan Saeed Doc Files In PDF
Khan Saeed Doc Files In PDF
 
Chart Of Accounts
Chart Of AccountsChart Of Accounts
Chart Of Accounts
 
Payton N Boyles
Payton N BoylesPayton N Boyles
Payton N Boyles
 

Similar to Measuring Sentiment Through NLP-Colored Glasses

Research uden at stille spørgsmål, Sophie Van Neck, InSites Consulting
Research uden at stille spørgsmål, Sophie Van Neck, InSites ConsultingResearch uden at stille spørgsmål, Sophie Van Neck, InSites Consulting
Research uden at stille spørgsmål, Sophie Van Neck, InSites ConsultingIBM Danmark
 
Making most of marketing dashboards
Making most of marketing dashboardsMaking most of marketing dashboards
Making most of marketing dashboardsStratigent
 
How Vistaprint Manages Customer Engagement
How Vistaprint Manages Customer EngagementHow Vistaprint Manages Customer Engagement
How Vistaprint Manages Customer EngagementOur Social Times
 
Breeding Loyalty Feb 2 2012
Breeding Loyalty Feb 2 2012Breeding Loyalty Feb 2 2012
Breeding Loyalty Feb 2 2012parvatimahadev
 
Listening to Data
Listening to DataListening to Data
Listening to DataMABSIV
 
Empirical Sentiment Accuracy Bounds
Empirical Sentiment Accuracy BoundsEmpirical Sentiment Accuracy Bounds
Empirical Sentiment Accuracy BoundsVisible Technologies
 
Digital Strategy, Simplicity through Complexity
Digital Strategy, Simplicity through ComplexityDigital Strategy, Simplicity through Complexity
Digital Strategy, Simplicity through ComplexityInsight Summit Series
 
Measuring User Delight
Measuring User DelightMeasuring User Delight
Measuring User DelightArun Antony
 
Mining and analyzing social media hicss 45 tutorial – part 2
Mining and analyzing social media hicss 45 tutorial – part 2Mining and analyzing social media hicss 45 tutorial – part 2
Mining and analyzing social media hicss 45 tutorial – part 2Dave King
 
Agile Metrics...That Matter
Agile Metrics...That MatterAgile Metrics...That Matter
Agile Metrics...That MatterErik Weber
 
Customer Experience Report Part 1
Customer Experience Report Part 1Customer Experience Report Part 1
Customer Experience Report Part 1davidmchattie
 
Video Game Usability Testing - Answering the Why
Video Game Usability Testing - Answering the WhyVideo Game Usability Testing - Answering the Why
Video Game Usability Testing - Answering the Whyaustinupa
 
Google Analytics Standard Presentation - GA Event, San Francisco 2011
Google Analytics Standard Presentation - GA Event, San Francisco 2011Google Analytics Standard Presentation - GA Event, San Francisco 2011
Google Analytics Standard Presentation - GA Event, San Francisco 2011Kayden Kelly
 
Brand tracking --_presentation brown forman_082212
Brand tracking --_presentation brown forman_082212Brand tracking --_presentation brown forman_082212
Brand tracking --_presentation brown forman_082212chouwinn
 
Some Things You Can't Wireframe
Some Things You Can't Wireframe Some Things You Can't Wireframe
Some Things You Can't Wireframe Wolf Becvar
 
Prediktiv analys och kundlojalitet
Prediktiv analys och kundlojalitetPrediktiv analys och kundlojalitet
Prediktiv analys och kundlojalitetIBM Sverige
 
Curso en Neuromarketing y comportamiento del consumidor
Curso en Neuromarketing y comportamiento del consumidorCurso en Neuromarketing y comportamiento del consumidor
Curso en Neuromarketing y comportamiento del consumidorAlexander Silva Lopera
 

Similar to Measuring Sentiment Through NLP-Colored Glasses (20)

Research uden at stille spørgsmål, Sophie Van Neck, InSites Consulting
Research uden at stille spørgsmål, Sophie Van Neck, InSites ConsultingResearch uden at stille spørgsmål, Sophie Van Neck, InSites Consulting
Research uden at stille spørgsmål, Sophie Van Neck, InSites Consulting
 
Making most of marketing dashboards
Making most of marketing dashboardsMaking most of marketing dashboards
Making most of marketing dashboards
 
How Vistaprint Manages Customer Engagement
How Vistaprint Manages Customer EngagementHow Vistaprint Manages Customer Engagement
How Vistaprint Manages Customer Engagement
 
Breeding Loyalty Feb 2 2012
Breeding Loyalty Feb 2 2012Breeding Loyalty Feb 2 2012
Breeding Loyalty Feb 2 2012
 
Listening to Data
Listening to DataListening to Data
Listening to Data
 
Empirical Sentiment Accuracy Bounds
Empirical Sentiment Accuracy BoundsEmpirical Sentiment Accuracy Bounds
Empirical Sentiment Accuracy Bounds
 
Digital Strategy, Simplicity through Complexity
Digital Strategy, Simplicity through ComplexityDigital Strategy, Simplicity through Complexity
Digital Strategy, Simplicity through Complexity
 
Measuring User Delight
Measuring User DelightMeasuring User Delight
Measuring User Delight
 
Mining and analyzing social media hicss 45 tutorial – part 2
Mining and analyzing social media hicss 45 tutorial – part 2Mining and analyzing social media hicss 45 tutorial – part 2
Mining and analyzing social media hicss 45 tutorial – part 2
 
Selling and doing good business
Selling and doing good businessSelling and doing good business
Selling and doing good business
 
Chap012
Chap012Chap012
Chap012
 
Agile Metrics...That Matter
Agile Metrics...That MatterAgile Metrics...That Matter
Agile Metrics...That Matter
 
Customer Experience Report Part 1
Customer Experience Report Part 1Customer Experience Report Part 1
Customer Experience Report Part 1
 
Video Game Usability Testing - Answering the Why
Video Game Usability Testing - Answering the WhyVideo Game Usability Testing - Answering the Why
Video Game Usability Testing - Answering the Why
 
Google Analytics Standard Presentation - GA Event, San Francisco 2011
Google Analytics Standard Presentation - GA Event, San Francisco 2011Google Analytics Standard Presentation - GA Event, San Francisco 2011
Google Analytics Standard Presentation - GA Event, San Francisco 2011
 
Brand tracking --_presentation brown forman_082212
Brand tracking --_presentation brown forman_082212Brand tracking --_presentation brown forman_082212
Brand tracking --_presentation brown forman_082212
 
Some Things You Can't Wireframe
Some Things You Can't Wireframe Some Things You Can't Wireframe
Some Things You Can't Wireframe
 
Prediktiv analys och kundlojalitet
Prediktiv analys och kundlojalitetPrediktiv analys och kundlojalitet
Prediktiv analys och kundlojalitet
 
Curso en Neuromarketing y comportamiento del consumidor
Curso en Neuromarketing y comportamiento del consumidorCurso en Neuromarketing y comportamiento del consumidor
Curso en Neuromarketing y comportamiento del consumidor
 
AI_4.pptx
AI_4.pptxAI_4.pptx
AI_4.pptx
 

Recently uploaded

Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxLoriGlavin3
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenHervé Boutemy
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxLoriGlavin3
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxLoriGlavin3
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxhariprasad279825
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersNicole Novielli
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024Lorenzo Miniero
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Manik S Magar
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Sample pptx for embedding into website for demo
Sample pptx for embedding into website for demoSample pptx for embedding into website for demo
Sample pptx for embedding into website for demoHarshalMandlekar2
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 3652toLead Limited
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 

Recently uploaded (20)

Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache Maven
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptx
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptx
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software Developers
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Sample pptx for embedding into website for demo
Sample pptx for embedding into website for demoSample pptx for embedding into website for demo
Sample pptx for embedding into website for demo
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 

Measuring Sentiment Through NLP-Colored Glasses

  • 1. Looking at the world through NLP-colored glasses Dr. John Feland, CEO, Argus Insights Sentiment Analysis Symposium Research Workshop 8 Nov 2011 Tweet it! @argusinsights
  • 2. If you can not measure it, you can not improve it. -Lord Kelvin © 2011 Argus Insights, Inc. Confidential: Do Not Distribute Page 2
  • 3. Always Check your math -my First Grade Teacher
  • 4. Trust, but Verify - Russian Proverb (Доверяй, но проверяй)
  • 5. Most Corporate Clients Treat Sentiment as a Black Box Raw musings of Numerical Market Magic consumers Sentiment Sentiment Analysis • Corporate Decision Makers blinded by precision • Using these numbers to drive performance • What is the impact of getting it wrong? © 2011 Argus Insights, Inc. Confidential: Do Not Distribute Page 5
  • 6. How we checked the math Unnamed sentiment provider • Over 1900 user reviews taken from the Tablet Market • Sources include most of the major online retailers • Leveraged best-in-class sentiment provider – Scored entire verbatim using tuned sentiment model – Usage is outside of vendor’s recommended use • Surprised at the results © 2011 Argus Insights, Inc. Confidential: Do Not Distribute Page 6
  • 7. Initial compared sentiment scores directly 0.6 Actual Distribution SV Distribution Percentage Distribution 0.5 0.4 0.3 0.2 0.1 0 -1 -0.75 -0.5 -0.25 0 0.25 0.5 0.75 1 Consumer Sentiment • All scales normalized (linearly) to -1 to +1 • SV Sentiment Score within Review Range 7% of the time • SV Sentiment within correct Positive, Neutral or Negative territory 51% of the time • SV results are biased towards the negative for Tablets © 2011 Argus Insights, Inc. Confidential: Do Not Distribute Page 7
  • 8. Scaled so that SV distro matches User distro 0.6 Actual Distribution SV Distribution P{ercentage of Distribution 0.5 0.4 0.3 0.2 0.1 0 -1 -0.75 -0.5 -0.25 0 0.25 0.5 0.75 1 Consumer Sentiment • Trimmed and scaled SV results to match actual distribution • Signficant performance improvements – Correct Bin: 33% -> almost 5X improvement – Correct Sign: 74% -> almost 50% improvement © 2011 Argus Insights, Inc. Confidential: Do Not Distribute Page 8
  • 9. Comparing actual to scored by product Reviews Raw SV Score Corrected SV Score 1 ASUS Eee Slate RIM PlayBook RIM PlayBook 2 Iconia Andr Tablet iPad 2 iPad 2 3 Motorola Xoom Iconia Andr Tablet Iconia Andr Tablet 4 iPad 2 Samsung Galaxy Tab 7 Motorola Xoom 5 RIM PlayBook Motorola Xoom Samsung Galaxy Tab 7 6 Samsung Galaxy Tab 7 ASUS Eee Slate ASUS Eee Slate 7 Iconia Win Tablet Iconia Win Tablet Iconia Win Tablet 8 HP TouchPad HP TouchPad HP TouchPad • Calculated sentiment could lead to the wrong competitive understanding • Selecting the wrong target can be disastrous for the enterprise • Responsibility to our clients demand transparency © 2011 Argus Insights, Inc. Confidential: Do Not Distribute Page 9
  • 10. New sentiment models - where the math checks out • Argus Insights uses customer reported sentiment • Proven predictor of market performance • Full transparency to clients, enabling them to trust and verify © 2011 Argus Insights, Inc. Confidential: Do Not Distribute Page 10
  • 11. With great power comes great responsibility -Voltaire © 2011 Argus Insights, Inc. Confidential: Do Not Distribute Page 11