Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How to use a Kalman Filter in Brand Tracking?

115 views

Published on

Sample variability makes it difficult to distinguish between noise and true shifts. Traditionally, we rely on rolling averages to smooth out the extremes. Kalman Filter offers a flexible alternative and allows us to incorporate prior/additional information related to the sample and the market environment.

Note, in statistics and control theory, Kalman filtering, also known as linear quadratic estimation (LQE), is an algorithm that uses a series of measurements observed over time, containing statistical noise and other inaccuracies, and produces estimates of unknown variables that tend to be more accurate than those based on a single measurement alone, by estimating a joint probability distribution over the variables for each timeframe. The filter is named after Rudolf E. KΓ‘lmΓ‘n, one of the primary developers of its theory. One of the first uses of Kalman Filter was in the Apollo navigation computer.

Published in: Education
  • Get paid to post comments on Facebook - $25 per hour ➀➀ http://t.cn/AieXiXbg
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

How to use a Kalman Filter in Brand Tracking?

  1. 1. October 2019 How to use a Kalman Filter in Brand Tracking? Jane Tang Webinar, 4 October 2019
  2. 2. #NewMR Sponsors Communication Gold Silver October 2019
  3. 3. How to use a Kalman Filter in Brand Tracking? Jane Tang jane.tang@bootstrapanalytics.com 3
  4. 4. FLY ME TO THE MOON: The Application Of Kalman Filter To Tracking Data Jane Tang, Andrew Grenville & Karen Buros SAMPLING IN THE RIVER TWITTER: A DIY Tool for Social Media Tracking via the REST API Jack Horne & Jane Tang Advanced Research Techniques Forum 2016 & 2017 4
  5. 5. - β€œSmooth” trajectory - Periodical observations - Random measurement errors - Simple and flexible tool to optimally separate the signal from the noise in the data 5 Tracking movement of a spacecraft going to the moon https://ieeexplore.ieee.org/document/5466132 Apollo Space Program Tracking perception of a brand in the minds of consumers Brand Tracking
  6. 6. How Kalman Filter Works http://bilgin.esme.org/BitsBytes/KalmanFilterforDummies.aspx http://www.cs.unc.edu/~tracker/media/pdf/SIGGRAPH2001_CoursePack_08.pdf !𝑋# = 𝐾# & 𝑋# + 1 βˆ’ 𝐾# & !𝑋#*+ estimation at time t Kalman gain measured value estimation at time (t – 1) State/prior equation ,π‘₯# = .π‘₯#*+ + 𝑀# β†’ 𝑝 𝑀 ~𝑁 0, 𝑄 Measurement/update equation .π‘₯# = π‘₯# + 𝑣# β†’ 𝑝 𝑣 ~𝑁 0, 𝑅 6
  7. 7. Equations Simplify to a Random Walk !𝑋# = 𝐾# & 𝑋# + 1 βˆ’ 𝐾# & !𝑋#*+ estimation at time t Kalman gain measured value estimation at time (t – 1) Time update β€œprediction” ,π‘₯# = .π‘₯#*+ (1) Prior for subsequent state (2) Prior error covariance 9𝑃# = 𝑃#*+ + 𝑄 external shock Measurement update β€œcorrection” random walk (1) Compute the Kalman gain 𝐾# = 9𝑃# 9𝑃# + 𝑅 *+ random measurement error (2) Update estimate .π‘₯# = ,π‘₯# + 𝐾# π‘₯# βˆ’ ,π‘₯# (3) Update error covariance 𝑃# = 9𝑃# 1 βˆ’ 𝐾#random walk 7
  8. 8. A Simple Example 𝑑 π‘₯# ,π‘₯# 9𝑃# Time update Measurement update .π‘₯# 𝑃# 1 0.390 0.000 1.000 ,π‘₯< = 0 9𝑃< = 1 𝐾# = 9𝑃# 9𝑃# + 𝑅 *+ =1 1 + 0.1 *+ = 0.909 .π‘₯# = ,π‘₯# + 𝐾# π‘₯# βˆ’ ,π‘₯# = 0 + 0.909 0.390 βˆ’ 0 = 0.355 𝑃# = 9𝑃# 1 βˆ’ 𝐾# =1 1 βˆ’ 0.909 = 0.091 0.355 0.091 2 0.500 0.355 0.092 ,π‘₯# = .π‘₯#*+ = 0.355 9𝑃# = 𝑃#*+ + 𝑄 = 0.092 𝐾# = 0.092 0.092 + 0.1 *+ = 0.479 .π‘₯# = 0.355 + 0.479 0.5 βˆ’ 0.355 = 0.424 𝑃# = 0.092 1 βˆ’ 0.479 = 0.048 0.424 0.048 3 0.480 0.424 0.049 0.443 0.033 4 0.290 0.443 0.034 0.404 0.025 𝑅 = 0.1 variance around observations (best guess) 𝑄 = 0.001 external shock variance 8
  9. 9. A Simple Example Value(x) Time (t) 9
  10. 10. Size of the Measurement Error (R) Value(x) Time (t) 10
  11. 11. Detecting Trend Value(x) Time (t) 11
  12. 12. 12 Β† We have large, long-term (6+ years) tracking programs, carefully controlled wave-to- wave, with trend lines like these. How clear is the story? Continuing to gain ground? Year 1.1Year 1.2Year 2.1 Year2.2 Year 3.1Year 3.2Year 4.1Year 4.2Year 5.1Year 5.2Year 6.1Year 6.2Year 7.1 Likelihood to Recommend Competitor 1 Competitor 2 Competitor 3 Competitor 4 Competitor 5 Competitor 6 Yr 1A Yr 1B Yr 2A Yr 2B Yr 3A Yr 3B Yr 4A Yr 4B Yr 5A Yr 5B Yr 6A Yr 6B Yr 7A Favorite Competitor 1 Competitor 2 Competitor 3 Competitor 4 Competitor 5 Competitor 6 Stabilizing? Losing steam? The Truth: I Dread Tracking Data
  13. 13. 13 First, let’s standardize the results to the baseline measurement Then we can apply the Kalman Filter 80 90 100 110 120 130 140 150 160 Year 1.1 Year 1.2 Year 2.1 Year2.2 Year 3.1 Year 3.2 Year 4.1 Year 4.2 Year 5.1 Year 5.2 Year 6.1 Year 6.2 Year 7.1 Kalman Estimate Likely to Recommend: Ratio to Baseline Competitor 1 Competitor 2 Competitor 3 Competitor 4 Competitor 5 Competitor 6 80 90 100 110 120 130 140 150 160 Year 1.1 Year 1.2 Year 2.1 Year2.2 Year 3.1 Year 3.2 Year 4.1 Year 4.2 Year 5.1 Year 5.2 Year 6.1 Year 6.2 Year 7.1 Observed Likely to Recommend: Ratio to Baseline Competitor 1 Competitor 2 Competitor 3 Competitor 4 Competitor 5 Competitor 6 What’s happening here? Can the Kalman Filter Clarify the Story?
  14. 14. 14 A More Detailed The Effect of the Filter? Β† Kalman Filter for Competitors 2 and 3 – Softening the extremes 80 90 100 110 120 130 140 150 160 Year 1.1 Year 1.2 Year 2.1 Year2.2 Year 3.1 Year 3.2 Year 4.1 Year 4.2 Year 5.1 Year 5.2 Year 6.1 Year 6.2 Year 7.1 Likelihood to Recommend: Competitors 2 and 3 Observed 2 Estimated 2 Observed 3 Estimated 3
  15. 15. 15 Taking a Look at Favorite Brand… We standardize the results to the baseline measurement Then apply the Kalman Filter 40 60 80 100 120 140 160 180 Year 1.1 Year 1.2 Year 2.1 Year2.2 Year 3.1 Year 3.2 Year 4.1 Year 4.2 Year 5.1 Year 5.2 Year 6.1 Year 6.2 Year 7.1 Observed Favorite: Ratio to Baseline Competitor 1 Competitor 2 Competitor 3 Competitor 4 Competitor 5 Competitor 6 40 60 80 100 120 140 160 180 Year 1.1 Year 1.2 Year 2.1 Year2.2 Year 3.1 Year 3.2 Year 4.1 Year 4.2 Year 5.1 Year 5.2 Year 6.1 Year 6.2 Year 7.1 Kalman Estimate Favorite: Ratio to Baseline Competitor 1 Competitor 2 Competitor 3 Competitor 4 Competitor 5 Competitor 6 Competitor 4 Suffers
  16. 16. 16 A More Detailed Look at Competitors 4 and 5 Β† Again, softening the extremes 40 60 80 100 120 140 160 180 Year 1.1 Year 1.2 Year 2.1 Year2.2 Year 3.1 Year 3.2 Year 4.1 Year 4.2 Year 5.1 Year 5.2 Year 6.1 Year 6.2 Year 7.1 Favorite: Competitors 4 and 5 Observed 4 Estimated 4 Observed 5 Estimated 5 KF provides a way to simultaneously account for β€˜time’ (as an alternative to β€˜three month rolling average’) and measurement error.
  17. 17. 17 Can We Apply This Approach to Social Media Data? Β† Social positive sentiment over the tracking period Β† Application of the Kalman Filter smooths the trend lines a bit. Greater variation across time is likely due to changing social media audience creating the β€˜sentiment’ values. 0.40 0.45 0.50 0.55 0.60 0.65 0.70 Year 1.1Year 1.2Year 2.1 Year2.2 Year 3.1Year 3.2Year 4.1Year 4.2Year 5.1Year 5.2Year 6.1Year 6.2Year 7.1 Social Media: Positive Sentiment Competitor 1 Competitor 2 Competitor 3 Competitor 4 Competitor 5 Competitor 6 0.40 0.45 0.50 0.55 0.60 0.65 0.70 Year 1.1 Year 1.2 Year 2.1 Year2.2 Year 3.1 Year 3.2 Year 4.1 Year 4.2 Year 5.1 Year 5.2 Year 6.1 Year 6.2 Year 7.1 Kalman Estimate: Positive Sentiment Competitor 1 Competitor 2 Competitor 3 Competitor 4 Competitor 5 Competitor 6
  18. 18. 18 Running Average vs. Kalman Filter Royal Bank, Olympic Sponsorship Awareness, Jan-Jun 2008 (pre), Jan-Jun 2010 (during), and Jan-Jun 2011 (post)
  19. 19. Incorporating Known System Instability 𝑑 π‘₯# ,π‘₯# 9𝑃# .π‘₯# 𝑃# 1 0.390 0.000 1.000 0.355 0.091 𝑅 = 0.1 variance around observations (best guess) 𝑄 = π‘£π‘Žπ‘Ÿπ‘–π‘Žπ‘π‘™π‘’ π‘ž# 0.001 2 0.500 0.355 0.092 0.424 0.0480.001 3 0.480 0.424 0.049 0.443 0.0330.001 4 0.290 0.443 0.034 0.404 0.0250.001 5 0.250 0.404 0.026 0.356 0.0310.020 6 0.320 0.356 0.031 0.340 0.0450.050 7 0.340 0.340 0.045 0.340 0.0590.100 8 0.480 0.340 0.059 0.397 0.0410.010 9 0.410 0.397 0.041 0.401 0.0300.001 9𝑃# = 𝑃#*+ + 𝑄 external shock 𝐾# = 9𝑃# 9𝑃# + 𝑅 *+ random measurement error 𝐾# 0.909 0.479 0.328 0.253 0.312 0.448 0.592 0.409 0.295 𝑃# = 9𝑃# 1 βˆ’ 𝐾# 19
  20. 20. Incorporating Known System Instability 0.000 0.100 0.200 0.300 0.400 0.500 0.600 1 2 3 4 5 6 7 8 9 10 Observed Estimated with System Shock Value(x) Time (t) Q 20
  21. 21. K 20K 40K 60K 19 26 2 9 16 23 2 9 16 23 30 6 13 20 27 4 11 18 25 1 NumberofTweetsmentioningβ€œtrump” 0 January 2017 February March April May Incorporating Twitter Brand Mention in Brand Tracking Twitter Brand mention is used as an indicator of system instability. β€œBuzz” brand equity volatility∝ 9𝑃# = 𝑃#*+ + 𝑄 June 21
  22. 22. Tracking the β€œTrump” Brand January 2017 February March April May Trumpapprovalrating(SBA) Total US Sample June 38% 40% 42% 44% 46% 48% 50% 25 27 30 3 6 10 15 17 27 1 10 13 15 17 22 27 29 5 7 12 19 21 24 26 28 5 8 17 24 2 7 Raw KF - Constant KF w Twitter Updates 22
  23. 23. β€œTrump” Brand Tracking in Subpopulation Trumpapprovalrating(SBA) Midwest Female January 2017 February March April May June 20% 30% 40% 50% 60% 25 27 30 3 6 10 15 17 27 1 10 13 15 17 22 27 29 5 7 12 19 21 24 26 28 5 8 17 24 2 7 Raw KF - Constant KF w Twitter Updates 23
  24. 24. THANK YOU! jane.tang@bootstrapanalytics.com 24
  25. 25. 25 Q & A Ray Poynter Chief Research Officer Potentiate Jane Tang Principal Bootstrap Analytics
  26. 26. #NewMR Sponsors Communication Gold Silver October 2019

Γ—